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EXECUTIVE  SUMMARY 


BACKGROUND 

The  Chemistry  Division  of  the  Naval  Researeh  Laboratory  has  partieipated  in  several  programs 
funded  by  SERDP  and  ESTCP  whose  goal  has  been  to  enhance  the  classification  ability  of  the 
Multi-sensor  Towed  Array  Detection  System  (MTADS).  The  process  has  been  based  on  making 
use  of  both  the  location  information  inherent  in  an  item’s  magnetometry  response  and  the  shape 
and  size  information  inherent  in  the  response  to  electromagnetic  induction  (EMI)  sensors.  To 
date,  most  of  these  systems  have  used  time-domain  EMI  (TEM)  sensors  with  the  notable 
exception  of  the  Geophex  GEM-3  frequency-domain  sensor.  In  past  efforts,  classification 
performance  has  been  limited  by  both  the  information  available  from  the  EMI  sensors  and  by 
signal-to-noise  limitations.  Three  of  the  largest  noise  terms  are  inherent  sensor  noise,  motion- 
induced  noise,  and  sensor  location  uncertainty. 

The  three  most  successful  demonstrations  to  date  of  EMI-based  discrimination  all  involved  cued 
detection  with  gridded  collection  of  EMI  data.  The  success  of  the  gridded  data  collections  was 
due  to  the  combination  of  minimal  location  uncertainty,  no  motion-induced  noise,  and  sufficient 
SNR.  The  downside  of  the  implementations  previously  demonstrated  is  that  they  were  relatively 
slow  and  inefficient,  especially  on  a  large  site.  The  TEMTADS  array  was  designed  to  combine 
the  data  quality  advantages  of  a  gridded  survey  with  the  coverage  efficiencies  of  a  vehicular 
system.  The  design  goal  of  this  system  was  to  collect  data  equal,  if  not  better,  in  quality  to  the 
best  gridded  surveys  (the  relative  position  and  orientation  of  the  sensors  being  known  better  than 
for  gridded  data)  while  prosecuting  many  more  targets  each  field  day. 

OBJECTIVES  OF  THE  DEMONSTRATION 

The  objective  of  this  demonstration  was  to  validate  the  performance  of  the  TEMTADS  platform 
through  two  blind  tests.  The  TEMTADS  was  evaluated  in  terms  of  both  classification 
performance  (e.g.  false  alarm  rejection)  and  appropriateness  for  fielding  {i.e.  production  rate, 
usability,  etc.).  The  first  demonstration  was  conducted  at  the  Aberdeen  Proving  Ground  (APG) 
Standardized  UXO  Test  Site.  The  second  demonstration  was  conducted  as  part  of  the  ESTCP 
UXO  Classification  Study  at  the  former  Camp  San  Euis  Obispo.  At  each  demonstration,  the  site 
had  been  blind  seeded  with  a  significant  number  of  intact,  inert  UXO  types  to  challenge  UXO 
classification  systems  and  methodologies. 

DEMONSTRATION  RESULTS 

The  raw  signature  data  from  the  TEMTADS  array  reflect  details  of  the  sensor/target  geometry  as 
well  as  inherent  EMI  response  characteristics  of  the  targets  themselves.  In  order  to  separate  out 
the  intrinsic  target  response  properties  from  sensor/target  geometry  effects  the  measured 
signature  is  inverted  to  estimate  principal  axis  magnetic  polarizabilities  using  a  standard  induced 
dipole  response  model.  The  performance  metrics  used  to  monitor  the  success  of  the  technology 
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relate  to  production  rate,  accuracy  of  inverted  features,  analysis  time,  correct  classification,  and 
ease  of  use. 

The  system  was  able  to  consistently  interrogate  125  or  more  cued  targets  per  day.  The  analysis, 
which  required  roughly  15  minutes  per  target,  resulted  in  a  false  alarm  reduction  by  over  50% 
with  95%  correct  identification  of  munitions.  The  average  error  in  predicted  location  was  less 
than  10cm  in  northing  and  easting  and  the  average  error  in  depth  estimation  was  less  than  5  cm 
for  non-overlapping  targets  with  reasonable  SNR.  Qualitatively,  the  TEMTADS  array  was 
found  to  be  easy  to  use  and  proved  to  be  a  robust  and  reliable  sensor  platform. 

IMPLEMENTATION  ISSUES 

Implementation  issues  for  this  system  /  technology  fall  into  two  categories:  operational  concerns 
and  data  quality  /  analysis  issues.  In  terms  of  operational  concerns,  the  TEMTADS  as 
implemented  is  a  large,  vehicle-towed  system  that  operates  best  in  large,  open  areas.  As  seen  at 
the  former  Camp  SEO  demonstration,  it  is  possible  to  operate  the  system  in  rocky  terrain  with 
grades  approaching  20%,  but  at  reduced  operating  capacity  and  increased  system  wear.  Smaller 
versions  of  the  system  are  currently  under  development  under  several  ESTCP  and  SERDP 
projects  to  address  these  concerns.  The  goal  is  to  design  and  field  units  more  amenable  to 
operation  in  more  confined  terrain  and  topology  and  smaller  tow  vehicles  /  man-portable  and 
handheld  operation.  Another  serious  limitation  is  anomaly  density.  Eor  all  sensors,  there  is  a 
limiting  anomaly  density  above  which  the  response  of  individual  targets  cannot  be  separated. 

We  have  chosen  relatively  small  sensors  for  this  array  which  should  help  with  this  problem  but 
we  cannot  eliminate  it.  Anomaly  densities  of  300  anomalies/acre  or  higher  would  limit  the 
applicability  of  this  system  as  more  than  20%  of  the  anomalies  would  have  another  anomaly 
within  a  meter.  In  terms  of  data  quality,  one  pays  a  small  penalty  in  signal  amplitude  to  use  the 
smaller  TEMTADS  sensor  coil  as  compared  to  other  systems  such  as  a  high-power  EM61  Mkll. 
The  dramatically  improved  electrical  performance  of  these  new  sensors  helps  counterbalance  this 
issue,  particularly  in  the  ability  to  perform  better  averaging  (or  stacking).  However,  one  needs  to 
have  in  place  robust  data  QC  techniques  to  know  when  to  employ  these  capabilities  in  an 
efficient  manner. 
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1.0 


INTRODUCTION 


1.1  BACKGROUND 

Unexploded  Ordnanee  (UXO)  deteetion  and  remediation  is  a  high  priority  triserviee  requirement. 
As  the  Defense  Seience  Board  reeently  wrote:  “Today’s  UXO  cleanup  problem  is  massive  in 
scale  with  some  10  million  acres  of  land  involved.  Estimated  cleanup  costs  are  uncertain  but  are 
clearly  tens  of  billions  of  dollars.  This  cost  is  driven  by  the  digging  of  holes  in  which  no  UXOs 
are  present.  The  instruments  used  to  detect  UXOs  (generally  located  underground)  produce 
many  false  alarms,  -  i.e.,  detections  from  scrap  metal  or  other  foreign  or  natural  objects  -  ,  for 
every  detection  of  a  real  unexploded  munition  found.”  [1] 

There  is  general  agreement  that  the  best  solution  to  the  false  alarm  problem  involves  the  use  of 
EMI  sensors  which,  in  principle,  allow  the  extraction  of  target  shape  parameters  in  addition  to  a 
size  and  depth  estimate.  We,  and  others,  have  fielded  systems  with  either  time-domain  or 
frequency-domain  EMI  sensors  with  the  goal  of  extracting  reliable  target  shape  parameters  and, 
thus,  improving  the  classification  capability  of  our  surveys.  In  practice,  the  classification  ability 
of  these  sensors  has  been  limited  by  signal-to-noise  limitations.  Three  of  the  largest  noise  terms 
are  inherent  sensor  noise,  motion-induced  noise,  and  sensor  location  uncertainty. 

The  three  most  successful  demonstrations  of  EMI-based  classification  all  involved  cued 
detection  with  gridded  collection  of  EMI  data  [2-4].  The  success  of  the  gridded  data  collections 
was  due  to  the  combination  of  minimal  location  uncertainty,  no  motion-induced  noise,  and 
sufficient  SNR.  The  downside  of  the  implementations  previously  demonstrated  is  that  they  were 
relatively  slow  and  inefficient,  especially  on  a  large  site.  We  have  constructed  an  EMI  sensor 
array  that  combines  the  classification  ability  of  a  gridded  survey  with  the  coverage  efficiency  of 
a  vehicular  array.  By  coming  to  a  stop  over  each  target  to  be  investigated  we  are  able  to  obtain 
all  the  benefits  of  a  gridded  survey  (negligible  relative  sensor  location  uncertainty,  no  motion- 
induced  noise,  and  high  SNR)  while  moving  rapidly  to  the  next  target  with  no  set-up  required 
gives  us  the  coverage  efficiency  required  for  practical  success. 

1.2  OBJECTIVE  OF  THE  DEMONSTRATION 

The  objective  of  this  demonstration  was  to  validate  the  technology  through  a  series  of  blind  test 
demonstrations.  We  conducted  a  shake-down  demonstration  of  the  technology  at  our  Blossom 
Point,  MD  field  site  but  a  blind  test  is  the  only  true  measure  of  system  performance.  The  first 
demonstration  was  conducted  at  the  Aberdeen  Proving  Ground  (APG)  Standardized  UXO  Test 
Site.  The  second  demonstration  was  conducted  as  part  of  the  ESTCP  UXO  Classification  Study 
at  the  former  Camp  San  Euis  Obispo.  At  each  demonstration,  the  site  had  been  blind  seeded 
with  a  significant  number  of  intact,  inert  UXO  types  to  challenge  UXO  classification  systems 
and  methodologies.  Demonstration  scoring  was  conducted  by  third  parties  to  maintain  the 
integrity  of  the  ground  truth  and  to  provide  an  unbiased  evaluation. 


Manuscript  approved  November  10,  2010. 
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1.3  REGULATORY  DRIVERS 


Stakeholder  aeceptance  of  the  use  of  olassifieation  teehniques  on  real  sites  will  require 
demonstration  that  these  teehniques  can  be  deployed  efficiently  and  with  high  probability  of 
discrimination.  The  first  step  in  this  process  was  to  demonstrate  acceptable  performance  on 
synthetic  test  sites  such  as  that  at  Aberdeen.  As  a  second  step,  demonstration  on  a  carefully 
prepared  and  blind-seeded  live  site  presented  a  more  real-world  scenario  while  providing 
sufficiently  complete  validation  data  to  accurately  determine  system  performance.  After  these 
hurdles  have  been  passed,  successful  demonstration  at  live  sites  will  further  facilitate  regulatory 
acceptance  of  the  UXO  classification  technology  and  methodology. 

2.0  TECHNOLOGY 

2.1  TECHNOLOGY  DESCRIPTION 

2.1.1  EMI  Sensors 

The  EMI  sensor  used  in  the  TEMTADS  array  is  based  on  the  Navy-funded  Advanced  Ordnance 
Locator  (AOL),  developed  by  G  &  G  Sciences.  The  AOL  consists  of  three  transmit  coils 
arranged  in  a  1-m  cube;  we  have  adopted  the  transmit  (Tx)  and  receive  (Rx)  subsystems  of  this 
sensor  directly,  converted  to  a  5  x  5  array  of  35  cm  sensors,  and  made  minor  modifications  to  the 
control  and  data  acquisition  computer  to  make  it  compatible  with  our  deployment  scheme. 

A  photograph  of  an  individual  sensor  element  under  construction  is  shown  in  the  left  panel  of 
Eigure  2-1.  The  transmit  coil  is  wound  around  the  outer  portion  of  the  form  and  is  35  cm  on  a 
side.  The  25-cm  receive  coil  is  wound  around  the  inner  part  of  the  form  which  is  re-inserted  into 
the  outer  portion.  An  assembled  sensor  with  the  top  and  bottom  caps  used  to  locate  the  sensor  in 
the  array  is  shown  in  the  right  panel  of  Eigure  2-1. 


Figure  2-1  -  Construction  details  of  an  individual  EMI  sensor  (left  panel)  and  the  assembled  sensor  with 
end  caps  attached  (right  panel). 
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Decay  data  are  collected  with  a  500  kHz  sample  rate  until  25ms  after  turn  off  of  the  excitation 
pulse.  This  results  in  a  raw  decay  of  12,500  points;  too  many  to  be  practical.  These  raw  decay 
measurements  are  grouped  into  115  logarithmically-spaced  “gates”  whose  center  times  are 
between  42  ps  to  25  ms  with  5%  widths  and  are  saved  to  disk.  Examples  of  the  measured 
transmit  pulse,  raw  decay,  and  gated  decay  are  shown  in  Figure  2-2. 


Figure  2-2  -  Measured  transmit  current  (on-time  upper  panel,  off-time 
second  panel),  full  measured  signal  decay  (third  panel),  and  gated  decay 
(fourth  panel)  as  discussed  in  the  text. 

2,1,2  Sensor  Array 

The  twenty- five  individual  sensors  are  arranged  in  a  5  x  5  array  as  shown  in  Figure  2-3.  The 
center-to-center  distance  is  40  cm  yielding  a  2  m  x  2  m  array.  Also  shown  in  Figure  2-3  is  the 
position  of  the  three  GPS  antennae  that  are  used  to  determine  the  location  and  orientation  of  the 
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array  for  each  cued  measurement.  A  picture  of  the  array  mounted  on  the  MTADS  EMI  sensor 
platform  is  shown  in  Figure  2-4. 


EM  Sensor 


GPS  Antenna 


Figure  2-3  -  Sketch  of  the  EMI  sensor  array  showing  the  position  of  the 
25  sensors  and  the  three  GPS  antennae. 


Figure  2-4  -  Sensor  array  mounted  on  the  MTADS  EMI  sensor  platform. 

2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  TEMTADS  array  was  designed  to  combine  the  data  quality  advantages  of  a  gridded  survey 
with  the  coverage  efficiencies  of  a  vehicular  system.  The  design  goal  of  this  system  was  to 
collect  data  equal,  if  not  better,  in  quality  to  the  best  gridded  surveys  (the  relative  position  and 
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orientation  of  the  sensors  being  known  better  than  for  gridded  data)  while  proseeuting  many 
more  targets  eaeh  field  day. 

There  are  obvious  limitations  to  the  use  of  this  teehnology.  The  array  is  a  2-m  square  so  fields 
where  the  vegetation  or  topography  interferes  with  passage  of  a  trailer  that  size  will  not  be 
amenable  to  the  use  of  the  present  array.  The  other  serious  limitation  will  be  anomaly  density. 
For  all  sensors,  there  is  a  limiting  anomaly  density  above  whieh  the  response  of  individual 
targets  eannot  be  separated.  We  have  ehosen  relatively  small  sensors  for  this  array  whieh  should 
help  with  this  problem  but  we  cannot  eliminate  it.  Based  on  experiments  at  our  test  pit  at 
Blossom  Point,  the  results  of  this  demonstration,  and  work  done  on  the  former  Camp  Sibert  data 
sets,  anomaly  densities  of  300  anomalies/acre  or  higher  would  limit  the  applicability  of  this 
system  as  more  than  20%  of  the  anomalies  would  have  another  anomaly  within  a  meter.  For  low 
SNR  targets,  our  standard  data  acquisition  parameters  may  not  be  sufficient.  The  system 
software  has  built  in  the  capability  to  vary  the  data  acquisition  parameters  on-the-fiy  based  on 
flags  in  the  target  file  and  can  be  reconfigured  manually  as  required.  One  area  in  need  of 
development  is  a  robust,  consolidated  data  collection  /  data  QC  methodology  for  determining 
when  there  is  a  low  SNR  anomaly  and  when  there  is  no  anomaly  present.  This  issue  is  an  area  of 
ongoing  research. 

3.0  PERFORMANCE  OBJECTIVES 

The  TEMTADS  array  was  deployed  to  demonstrations  at  both  the  APG  Standardized  UXO  Test 
Site  and  the  ESTCP  UXO  Classification  Study  site  at  the  former  Camp  SEO.  Due  to  the  nature 
of  the  demonstrations  at  each  site  and  the  employed  scoring  methods,  separate  Performance 
Objectives  were  used  for  each  demonstration,  as  documented  in  the  individual  demonstration 
reports  [5,6]  and  the  project  final  report  [7].  The  Performance  Objectives  for  the  two 
demonstrations  have  been  grouped  into  a  unified  collection  given  in  Table  3-1  due  to  significant 
overlap.  Please  refer  to  the  references  for  the  specific  objectives  for  each  demonstration.  Since 
the  TEMTADS  array  is  a  discrimination  technology,  the  performance  objectives  focus  on  the 
second  step  of  the  UXO  survey  problem;  we  assume  that  the  anomalies  from  all  targets  of 
interest  have  been  detected  and  included  on  the  target  list  that  we  worked  from. 

3.1  OBJECTIVE:  SITE  COVERAGE 

Each  demonstration  commenced  with  a  list  of  previously  identified  anomalies,  whose  locations 
were  determined  using  some  other  data.  The  expectation  of  the  demonstration  was  to  gather 
cued  data  with  the  TEMTADS  system  over  each  assigned  anomaly. 

3.1.1  Metric 

Site  coverage  is  defined  as  the  fraction  of  the  assigned  anomalies  surveyed  by  the  TEMTADS. 
Exceptions  were  made  for  topology  /  vegetation  interferences.  This  is  particularly  true  for 
demonstrations  where  the  footprints  of  the  detection  and  classification  systems  are  significantly 
different  (e.g.  EM61  Mkll  cart  and  TEMTADS). 
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Table  3-1  -  Performance  Objectives  for  the  Demonstrations 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Results 

Quantitative  Performance  Objectives 

Site  Coverage 

Fraction  of  assigned 
anomalies  interrogated 

Survey  results 

1 00%  as  allowed  for  by 
topography  /  vegetation 

Yes 

Calibration  Strip 
Results 

System  response 
consistently  matches 
physics-based  model 

System  response 
curves 

Daily  calibration 
strip  data 

<15%  rms  variation  in 
amplitude 

Down-track  location 
±  25  cm 

All  response  values  fall 
within  bounding  curves 

No 

Reduction  of 
False  Alarms 

Number  of  false  alarms 
eliminated  at 
demonstrator  operating 
point. 

Prioritized  dig  list 

Scoring  report  from 
APG 

Reduction  of  false  alarms 
by  >  50%  with  95% 
correct  identification  of 
munitions 

Yes 

Location 

Accuracy 

Average  error  and 
standard  deviation  in 
both  axes  for 
interrogated  items 

Estimated  location 
from  analysis 

Scoring  reports 

AN  and  AE  <10  cm 
aN  and  aE  <  1 5  cm 

No 

Depth  Accuracy 

Standard  deviation  in 
depth  for  interrogated 
items 

Estimated  location 
from  analyses 

Ground  truth  from 
validation  effort 

ADepth  <  5  cm 

aDepth  <10  cm 

No 

Production  Rate 

Number  of  targets 
interrogated  each  day 

Eog  of  field  work 

75  targets  per  day 

Yes 

Data  Throughput 

Throughput  of  data  QC 
process 

Eog  of  analysis 
work 

All  data  QC’ed  on  site 
and  at  pace  with  survey 

Yes 

Analysis  Time 

Average  time  required 
for  inversion  and 
classification 

Eog  of  analysis 
work 

1 5  min  per  target 

Yes 

Qualitative  Performance  Objective 

Ease  of  Use 

Feedback  from 
operator  on  ease  of 
use 

No  significant 
operational  issues 
identified  by  operator 

Yes 
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3.1,2  Data  Requirements 


The  collected  data  were  compared  to  the  original  anomaly  list.  Any  interferences  were  noted  in 
the  field  log  book  as  they  occurred. 

3.1.3  Success  Criteria 

The  objective  is  considered  met  if  100%  of  the  assigned  anomalies  are  surveyed  with  the 
exception  of  anomalies  located  in  areas  that  cannot  be  surveyed  due  to  topology  /  vegetation 
interferences. 

3.1.4  Results 

This  objective  was  successfully  met.  All  assigned  anomalies  at  APG  were  successfully 
investigated.  At  the  former  Camp  SLO,  all  but  five  of  the  assigned  anomalies  were  measured. 
Failure  to  measure  these  5  anomalies  was  due  to  the  presence  of  rocks  which  prevented  the 
operator  from  positioning  the  TEMTADS  over  the  target. 

3.2  OBJECTIVE:  CALIBRATION  STRIP  RESULTS 

This  objective  supports  that  each  sensor  system  is  in  good  working  order  and  collecting 
physically  valid  data  each  day  and  only  applies  to  the  former  Camp  SLO  demonstration.  The 
calibration  strip  was  surveyed  twice  daily.  The  peak  positive  response  of  each  emplaced  item 
from  each  run  was  compared  to  the  physics-based  response  curves  generated  prior  to  data 
collection  on  site  using  each  item  of  interest. 

3.2.1  Metric 

The  reproducibility  of  the  measured  response  of  each  sensor  system  to  the  items  of  interest  and 
the  comparison  of  the  response  to  the  response  predicted  by  the  physics-based  model  defines  this 
metric. 

3.2.2  Data  Requirements 

Response  curves  for  each  sensor  /  item  of  interest  pair  were  used  to  document  what  the  physics- 
based  response  of  the  system  to  the  item  should  be.  The  tabulated  peak  response  values  from 
each  survey  of  the  Calibration  Strip  were  used  to  demonstrate  the  reproducibility  and  validity  of 
the  sensor  readings. 

3.2.3  Success  Criteria 

The  objective  will  be  considered  met  if  all  measured  responses  fall  within  the  range  of  physically 
possible  values  based  on  the  appropriate  response  curve.  Additionally,  the  RMS  variation  in 
responses  should  be  less  than  15%  of  the  measured  response  and  the  down- track  location  of  the 
anomaly  should  be  within  25  cm  of  the  corresponding  seeded  item’s  true  location. 
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3.2,4  Results 


This  objective  was  not  successfully  met  in  full.  The  measured  peak  signals  for  all  of  the 
emplaced  items  generally  fit  well  within  the  physics-based  response  bounding  curves,  with  the 
4.2-in  mortar  and  the  shotput  results  giving  the  poorest  match,  with  a  tendency  to  underestimate 
the  peak  value.  Careful  examination  of  the  data  shows  that  this  variation  is  the  result  of  the  shot- 
to-shot  precision  with  which  the  array  was  positioned  in  exactly  the  same  spot  each  time. 
Because  the  response  curves  are  generated  assuming  the  target  is  directly  below  the  sensor,  any 
offset  in  the  sensor  position  will  result  in  the  derived  peak  signal  being  smaller  than  that 
predicted  by  the  curve,  as  is  observed.  Due  to  the  large  footprint  of  the  TEMTADS  array  and 
number  of  sensor  elements  contained  in  the  array,  the  array  is  considered  to  have  properly 
positioned  from  a  field  work  prospective  for  a  measurement  if  the  array  center  is  within  20  cm  of 
the  target  location.  This  does  not  impact  the  values  of  the  inverted  parameters  and  offers  the 
vehicle  operator  some  flexibility  in  the  field.  It  does  however  effect  the  measured  peak  signal 
amplitudes.  For  future  demonstrations,  the  metric  of  fitted  polarizability  amplitude  is 
recommended  as  a  replacement  metric  as  these  values  are  invariant  to  array  position. 

3.3  OBJECTIVE:  REDUCTION  OF  FALSE  ALARMS 

This  is  the  primary  measure  of  the  effectiveness  of  this  technology.  By  collecting  high-quality, 
precisely-located  data,  we  expect  to  be  able  to  discriminate  munitions  from  scrap  and  frag  with 
high  efficiency.  This  metric  was  not  part  of  the  performance  criteria  for  the  former  Camp  SLO 
demonstration,  so  it  was  only  evaluated  for  the  APG  demonstration. 

3,3,1  Metric 

At  a  seeded  test  site  such  as  the  APG  standardized  test  site,  the  metric  for  false  alarm  elimination 
is  straightforward.  We  prepared  a  ranked  dig  list  for  the  targets  we  interrogated  with  a  dig/no¬ 
dig  threshold  indicated  and  ATC  personnel  used  their  automated  scoring  algorithms  to  assess  our 
results. 


3.3.2  Data  Requirements 

The  identification  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators.  Our 
ranked  dig  list  was  the  input  for  this  objective  and  ATC’s  standard  scoring  is  the  output. 

3.3.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  more  than  50%  of  the  non-munitions  items  were 
labeled  as  no-dig  while  retaining  95%  of  the  munitions  items  on  the  dig  list. 

3.3.4  Results 

This  Objective  was  successfully  met.  The  TEMTADS  surveyed  anomalies  detected  by  the 
MTADS  magnetometer  system  in  the  Blind  Grid  and  Indirect  Fire  Areas.  Efficiency  (E)  and 
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false  positive  rejection  rate  (Rfp)  are  used  to  score  discrimination  performance  ability  at  two 
specific  operating  points  on  a  ROC  curve:  one  at  the  point  where  no  decrease  in  Pd  is  incurred 
and  the  other  at  the  operator-selected  threshold.  Efficiency  is  defined  as  the  fraction  of  detected 
ordnance  correctly  classified  as  ordnance  and  the  false  positive  rejection  rate  is  defined  as  the 
fraction  of  detected  clutter  correctly  classified  as  clutter.  The  results  for  the  Blind  Grid  and 
Indirect  Fire  Test  Areas  were  E=  0.99  and  Rfj,  =  0.99,  and  E=  0.98  and  Rfj,  =  0.92  at  the  operating 
point,  respectively.  With  no  loss  of  Pd,  the  results  were  E=  1.00  and  Rfp  =  0.95,  and  E=  1.00  and 
Rfp  =  0.58,  for  the  Blind  Grid  and  the  Indirect  Fire  Areas  respectively.  These  data  are 
summarized  from  Tables  7a  and  7c  of  Reference  8. 

3.4  OBJECTIVE:  LOCATION  ACCURACY 

An  important  measure  of  how  efficiently  any  required  remediation  will  proceed  is  the  accuracy 
of  predicted  location  of  the  targets  marked  to  be  dug.  Earge  location  errors  lead  to  confusion 
among  the  UXO  technicians  assigned  to  the  remediation  costing  time  and  often  leading  to 
removal  of  a  small,  shallow  object  when  a  larger,  deeper  object  was  the  intended  target. 

3.4.1  Metric 

The  average  error  and  deviation  in  both  horizontal  axes  was  computed  for  the  items  which  are 
selected  for  excavation  during  the  validation  phase  of  the  demonstration.  We  provided  an 
estimated  position  for  all  targets  we  interrogated  to  the  appropriate  personnel  at  each  site,  and 
they  used  their  scoring  algorithms  to  assess  our  results.  At  APG,  all  of  the  items  were  emplaced 
and  the  locations  are  known.  Therefore  the  items  were  not  excavated.  Aggregate  results  for  the 
APG  demonstration  were  provided  by  ATC. 

3.4.2  Data  Requirements 

The  location  of  most  of  the  items  in  the  test  field  is  known  to  the  appropriate  personnel  at  each 
site.  Our  dig  list  was  the  input  for  this  objective  and  a  standard  scoring  report  is  the  output. 

3.4.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  the  average  position  error  (low  bias)  and  the 
standard  deviation  (accurate  location)  in  both  dimensions  were  less  than  10  and  15  cm, 
respectively  for  APG,  and  less  than  5  and  10  cm,  respectively  for  the  former  Camp  SEO. 

3.4.4  Results 

This  Objective  was  not  successfully  met  in  full.  For  APG,  the  location  accuracy  of  fit 
parameters  generated  from  the  TEMTADS  array  data,  taken  from  Tables  9a  and  9c  of  Reference 
8,  are  within  5  cm  horizontally  (la)  and  6  cm  vertically  (la)  for  the  Indirect  Fire  Area. 
Horizontal  errors  are  not  calculated  for  the  Blind  Grid.  The  vertical  accuracy  was  4  cm  (la)  for 
the  Blind  Grid.  For  the  former  Camp  SEO,  the  average  Northing  position  error  for  all  measured 
data  was  1.5  cm,  while  the  average  Easting  position  error  was  3  cm.  The  standard  deviations. 
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however,  were  larger  than  desired,  with  both  values  about  25  cm.  Excluding  those  anomalies  for 
which  multiple  targets  were  found  produces  negligible  improvement.  We  suspect  the  higher 
values  are  due  to  the  large  number  of  small,  low  SNR  clutter  items,  which  result  in  greater 
uncertainty  in  both  the  measured  and  fitted  values.  Indeed,  if  we  restrict  ourselves  to  those 
anomalies  which  we  classified  as  likely  UXO,  the  Northing  and  Easting  standard  deviations  drop 
to  7cm  and  5  cm,  respectively. 

3.5  OBJECTIVE:  DEPTH  ACCURACY 

An  important  measure  of  how  efficiently  any  required  remediation  will  proceed  is  the  accuracy 
of  predicted  depth  of  the  targets  marked  to  be  dug.  Earge  depth  errors  lead  to  confusion  among 
the  UXO  technicians  assigned  to  the  remediation  costing  time  and  often  leading  to  removal  of  a 
small,  shallow  object  when  a  larger,  deeper  object  was  the  intended  target. 

3.5.1  Metric 

The  standard  deviation  of  the  predicted  depths  with  respect  to  the  ground  truth  was  computed  for 
the  items  which  were  selected  for  excavation  during  the  validation  phase  of  the  former  Camp 
SEO  study.  At  APG,  all  of  the  items  were  emplaced  and  the  locations  are  known.  Therefore  the 
items  were  not  excavated.  Aggregate  results  for  the  APG  demonstration  were  provided  by  ATC. 

3.5.2  Data  Requirements 

The  anomaly  fit  parameters  and  the  ground  truth  for  the  excavated  items  were  required  to 
determine  the  performance  of  the  fitting  routines  in  terms  of  the  predicted  depth  accuracy. 

3.5.3  Success  Criteria 

This  objective  will  be  considered  as  met  if  the  average  error  in  depth  was  less  than  5  cm  and  the 
standard  deviation  was  less  than  10  cm. 

3.5.4  Results 

This  objective  was  not  met  successfully  in  full.  Eor  APG,  the  accuracy  of  the  depth  fit  parameter 
generated  from  the  TEMTADS  array  data,  taken  from  Tables  9a  and  9c  of  Reference  8,  was 
within  6  cm  vertically  (la)  for  the  Indirect  Eire  Area.  The  vertical  accuracy  was  4  cm  (la)  for 
the  Blind  Grid.  Eor  the  SEO  demonstration,  the  average  depth  error  for  all  measured  data  was 
2.5cm.  The  standard  deviation  was  a  bit  larger  than  desired,  with  a  value  of  14cm.  Excluding 
those  anomalies  for  which  multiple  targets  were  found  and  restricting  the  analysis  to  those 
anomalies  classified  as  ‘likely  UXO’  to  exclude  low  SNR  clutter  items,  the  standard  deviation 
reduces  to  7cm. 
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3.6  OBJECTIVE:  PRODUCTION  RATE 


Even  if  the  performanee  of  the  teehnology  for  the  metries  listed  above  is  satisfaetory,  there  is  an 
eeonomic  metrie  to  eonsider.  There  is  a  known  eost  of  remediating  a  suspeeted  munitions  item. 
If  the  cost  to  interrogate  a  target  is  greater  than  this  cost,  the  technology  will  be  useful  only  at 
sites  with  special  conditions  or  target  values.  Note,  however,  that  in  its  ultimate  implementation 
this  technology  will  result  in  reacquisition,  cued  interrogation,  and  target  flagging  in  one  visit  to 
the  site. 

3.6.1  Metric 

The  number  of  targets  interrogated  per  day  was  the  metric  for  this  objective.  Combined  with  the 
daily  operating  cost  of  the  technology  this  gives  the  per-item  cost. 

3.6.2  Data  Requirements 

The  metric  was  determined  from  the  combination  of  available  field  logs  and  the  survey  results. 
The  field  logs  record  the  amount  of  time  per  day  spent  acquiring  the  data  and  the  survey  results 
determine  the  number  of  anomalies  investigated  in  that  time  period. 

3.6.3  Success  Criteria 

For  the  APG  demonstration,  the  objective  was  considered  to  be  met  if  at  least  75  targets  were 
interrogated  each  survey  day.  For  the  former  Camp  SFO  demonstration,  the  production  rate 
target  was  125  anomalies/day. 

3.6.4  Results 

This  Objective  was  successfully  met.  At  APG,  214  anomalies  were  investigated  on  the  Blind 
Grid  over  the  course  of  1.43  work  days,  or  on  average  149  anomalies  /  work  day.  694  anomalies 
were  investigated  in  the  Indirect  Fire  Area  over  the  course  of  32  hours  and  30  minutes,  or  on 
average  170  anomalies  /  work  day.  At  the  former  Camp  SFO,  a  total  of  1547  anomalies 
(including  redo’s)  were  measured  over  a  10-day  period  for  an  average  of  155  anomalies/day. 

3.7  OBJECTIVE:  DATA  THROUGHPUT 

The  collection  of  a  complete,  high-quality  data  set  with  the  sensor  platform  is  critical  to  the 
downstream  success  of  any  UXO  classification  effort.  This  objective  considers  one  of  the  key 
data  quality  issues,  the  ability  of  the  data  analysis  workflow  to  support  the  data  collection  effort 
in  a  timely  fashion.  To  maximize  the  efficient  collection  of  high  quality  data,  a  series  of 
MTADS  standard  data  quality  checks  are  conducted  during  and  immediately  after  data  collection 
on  site.  Data  which  pass  the  QC  screen  are  then  processed  into  archival  data  stores.  Individual 
anomaly  analyses  are  then  conducted  on  those  archival  data  stores.  The  data  QC  /  preprocessing 
portion  of  the  workflow  needs  to  keep  pace  with  the  data  collection  effort  for  best  performance. 
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3.7,1  Metric 


The  throughput  of  the  data  quality  control  workflow  was  at  least  as  fast  the  data  collection 
process,  providing  real  time  feedback  to  the  data  collection  team  of  any  issues. 

3.7.2  Data  Requirements 

The  data  analysts  log  books  provided  the  necessary  data  for  determining  the  success  of  this 
metric. 

3.7.3  Success  Criteria 

This  objective  will  be  considered  met  if  all  collected  data  can  be  processed  through  the  data 
quality  control  portion  of  the  workflow  in  a  timely  fashion. 

3.7.4  Results 

This  objective  was  successfully  met.  Data  were  normally  downloaded  several  times  during  each 
workday,  and  quality  control  on  these  datasets  was  usually  completed  on  the  same  day.  Quality 
control  checks  successfully  caught  missed  anomalies,  a  small  number  of  corrupt  data  files,  and 
targets  which  needed  re-measuring. 

For  low  SNR  targets,  our  standard  data  acquisition  parameters  may  not  be  sufficient.  The  system 
software  has  built  in  the  capability  to  vary  the  data  acquisition  parameters  on-the-fly  based  on 
flags  in  the  target  file  and  can  be  reconfigured  manually  as  required.  To  date  these  capabilities 
have  not  been  demonstrated  and  could  potentially  have  an  impact  on  data  throughput.  A  robust, 
consolidated  data  collection  /  data  QC  methodology  for  determining  when  there  is  a  low  SNR 
anomaly  present  and  when  there  is  no  anomaly  present  is  necessary  to  accurately  and  efficiently 
utilize  these  capabilities.  This  issue  is  an  area  of  ongoing  research. 

3.8  OBJECTIVE:  ANALYSIS  TIME 

The  ultimate  implementation  of  this  technology  will  involve  on-the-fly  analysis  and 
classification.  The  time  for  this  will  be  limited  to  the  driving  time  to  the  next  anomaly  on  the 
list.  We  will  track  the  near-real-time  analysis  time  in  this  demonstration. 

3.8.1  Metric 

The  time  required  for  inversion  and  classification  per  anomaly  was  the  metric  for  this  objective 

3.8.2  Data  Requirements 

Analysis  time  was  determined  from  a  review  of  the  data  analysis  logs. 


12 


3.8,3  Success  Criteria 


The  objective  will  be  considered  to  be  met  if  the  average  inversion  and  classification  time  was 
less  than  15  min. 

3,8,4  Results 

This  Objective  was  successfully  met.  The  average  inversion  time  per  target  was  approximately 
2.5  minutes  on  our  field  laptop  computer.  The  total  average  analysis  time  amounted  to  12.5 
minutes  per  anomaly.  For  these  extensive  tests  of  the  system  in  field  mode,  we  took  the 
opportunity  to  consider  various  discrimination  and  classification  methods,  some  of  which  proved 
unfruitful.  As  a  result  of  lessons  learned  from  this  undertaking,  we  expect  the  average  analysis 
time  for  future  field  runs  to  be  less. 

3.9  OBJECTIVE:  EASE  OF  USE 

This  qualitative  objective  is  intended  as  a  measure  of  the  long-term  usability  of  the  technology. 
If  the  operator  does  not  report  that  the  technology  is  easy  to  use,  shortcuts  that  can  compromise 
the  efficiency  of  the  technology  will  begin  to  creep  into  daily  operations. 

3.9.1  Data  Requirements 

This  objective  was  evaluated  based  on  operator  feedback. 

3.9.2  Results 

This  Objective  was  successfully  met.  Based  on  vehicle  operator  feedback,  there  were  no 
significant  limitations  to  the  efficient  use  of  the  system  in  the  field.  Several  suggestions  were 
made  for  additional  improvements  to  the  navigation  and  data  collection  software.  They  have 
been  subsequently  incorporated. 

4.0  SITE  DESCRIPTION 

Two  demonstrations  were  conducted  for  this  project.  The  first  was  conducted  at  the 
Standardized  UXO  Test  Site  located  at  the  Aberdeen  Proving  Ground,  MD  during  the  period  of 
May  through  June,  2008.  The  second  was  conducted  at  the  former  Camp  San  Luis  Obispo,  CA 
ESTCP  UXO  Classification  Study  Demonstration  Site  in  June,  2009. 

4.1  APG  STANDARDIZED  UXO  TEST  SITE 

4.1.1  SITE  LOCATION  AND  HISTORY 

The  Standardized  UXO  Test  Site  is  located  adjacent  to  the  Trench  Warfare  facility  at  the 
Aberdeen  Proving  Ground.  The  specific  area  was  used  for  a  variety  of  ordnance  tests  over  the 
years.  Initial  magnetometer  and  EMI  surveys  conducted  by  the  MTADS  team  performed  after  a 
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“mag  and  flag”  survey  of  the  same  area  identified  over  a  thousand  remaining  anomalies.  These 
data  were  used  for  a  final  elean  up  of  the  site  prior  to  the  emplaeement  of  the  original  test  items. 
Prior  to  the  two  subsequent  reeonfiguration  events,  unexplained  anomalies  identified  by 
demonstrators  using  the  site  were  also  investigated  and  removed. 

This  was  the  site  of  our  first  field  demonstration  of  this  eombination  of  EMI  sensors  and  survey 
mode.  The  APG  site  is  loeated  elose  to  our  base  of  operations  in  southern  Maryland  and 
therefore  minimizes  the  logisties  costs  of  the  deployment.  Use  of  this  site  allowed  us  to  receive 
validation  results  from  near-real-world  conditions  without  incurring  the  logistics  and  intrusive 
investigation  expenses  that  would  be  required  for  a  demonstration  at  a  live  site. 

4.1.2  SITE  TOPOGRAPHY  AND  GEOLOGY 

According  to  the  soils  survey  conducted  for  the  entire  area  of  APG  in  1998,  the  test  site  consists 
primarily  of  Elkton  Series  type  soil  [9].  The  Elkton  Series  consist  of  very  deep,  slowly 
permeable,  poorly  drained  soils.  These  soils  formed  in  silty  aeolin  sediments  and  the  underlying 
loamy  alluvial  and  marine  sediments.  They  are  on  upland  and  lowland  fiats  and  in  depressions 
of  the  Mid-Atlantic  Coastal  Plain.  Slopes  range  from  0  to  2  percent. 

Overall,  the  demonstration  site  is  relatively  fiat  and  level.  There  are  some  low-lying  areas  in  the 
northwest  portion  of  the  site  that  tend  to  have  standing  water  during  the  wet  periods  of  the  year. 
The  current  sensor  system  is  not  sufficiently  weatherproofed  to  operate  through  standing  water. 
However,  during  the  most  recent  reconfiguration,  the  areas  most  prone  to  being  underwater  were 
excluded  from  the  survey  scenarios.  Anomalies  that  were  located  underwater  or  nearby  to  water 
at  the  time  of  survey  were  deferred  until  the  end  of  the  survey  and  were  interrogated  by  carefully, 
if  less  efficiently,  maneuvering  the  array  into  position. 

4.1.3  MUNITIONS  CONTAMINATION 

The  area  currently  occupied  by  the  site  has  seen  an  extensive  history  of  munitions  use.  As  an 
example,  in  2003  we  conducted  a  magnetometer  survey  of  a  previously  unremediated  area 
directly  adjacent  to  the  site  [10].  In  a  survey  area  of  approximately  1  hectare,  we  identified  2479 
anomalies,  of  which  1921  were  amenable  to  a  model  fit  using  our  standard  analysis.  Historical 
records  provided  by  ATC  and  previous  remediation  results  indicated  that  the  likely  munitions  of 
interest  for  this  site  were: 

•  Grenades,  Mkl,  Mkll,  and  Erench  VB  Rifle  w/o  chute 

•  Grenades,  Erench  VB  Rifle  w/  chute 

•  60mm  mortars  (including  2”  Smoke) 

•  3”  Stokes  (Smoke  and  HE) 

•  105  mm  projectiles 

•  155  mm  projectiles 
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4.1.4  SITE  CONFIGURATION 


Figure  4-1  is  a  map  of  the  Standardized  UXO  Test  Site  at  APG.  The  Calibration  and  Blind  Grids 
are  shown  along  with  the  various  Open  Field  Areas. 


Figure  4-1  -  Map  of  the  reconfigured  APG  Standardized  UXO  Test  Site. 
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4.2  FORMER  CAMP  SAN  LUIS  OBISPO 


4.2.1  SITE  LOCATION  AND  HISTORY 

The  site  deseription  material  reprodueed  here  is  taken  from  the  reeent  SI  report  [11],  More 
details  ean  be  obtained  in  the  report.  The  former  Camp  SLO  is  approximately  2,101  acres 
situated  along  Highway  1,  approximately  five  miles  northwest  of  San  Luis  Obispo,  California. 
The  majority  of  the  area  consists  of  mountains  and  canyons.  The  site  for  this  demonstration  is  a 
mortar  target  on  hilltop  in  MRS  05  (within  former  Rifle  Range  #12). 

Camp  SLO  was  established  in  1928  by  the  State  of  California  as  a  National  Guard  Camp. 
Identified  at  that  time  as  Camp  Merriam,  it  originally  consisted  of  5,800  acres.  Additional  lands 
were  added  in  the  early  1940s  until  the  acreage  totaled  14,959.  From  1943  to  1946,  Camp  SLO 
was  used  by  the  U.S.  Army  for  infantry  division  training  including  artillery,  small  arms,  mortar, 
rocket,  and  grenade  ranges.  Following  the  end  of  World  War  II,  a  small  portion  of  the  former 
camp  land  was  returned  to  its  former  private  owners.  The  U.S.  Army  was  making  arrangements 
to  relinquish  the  rest  of  Camp  SLO  to  the  State  of  California  and  other  government  agencies 
when  the  conflict  in  Korea  started  in  1950.  The  camp  was  reactivated  at  that  time. 

The  U.S.  Army  used  the  former  camp  during  the  Korean  War  from  1951  through  1953  where  the 
Southwest  Signal  Center  was  established  for  the  purpose  of  signal  corps  training.  Following  the 
Korean  War,  the  camp  was  maintained  in  inactive  status  until  it  was  relinquished  by  the  Army  in 
the  1960s  and  1970s.  Approximately  4,685  acres  was  relinquished  to  the  General  Services 
Administration  (GSA)  in  1965.  GSA  then  transferred  the  property  to  other  agencies  and 
individuals  beginning  in  the  late- 1960s  through  the  1980s;  most  of  which  was  transferred  for 
educational  purposes  (California  Polytechnic  State  University  and  Cuesta  College).  A  large 
portion  of  Camp  SLO  (the  original  5,880  acres)  has  been  retained  by  the  California  National 
Guard  (CNG)  and  is  not  part  of  the  FUDS  program. 

This  site  was  chosen  as  the  second  in  a  progression  of  increasingly  more  complex  sites  for 
demonstration  of  the  classification  process  as  part  of  the  ESTCP  UXO  Classification  Study.  The 
first  site  in  the  series,  former  Camp  Sibert,  had  only  one  target-of-interest  and  item  “size”  was  an 
effective  discriminant.  At  this  site,  there  are  at  least  four  targets-of-interest;  60-mm,  81 -mm,  and 
4.2-in  mortars  and  2.36-in  rockets.  This  introduces  another  layer  of  complexity  into  the  process. 

4.2.2  SITE  TOPOGRAPHY  AND  GEOLOGY 

The  former  Camp  SLO  site  consists  mainly  of  mountains  and  canyons  classified  as  grassland, 
wooded  grassland,  woodland,  or  brush.  A  major  portion  of  the  site  is  identified  as  grassland  and 
is  used  primarily  for  grazing.  Los  Padres  National  Forest  (woodland)  is  located  to  the  north- 
northeastern  portion  of  the  site.  During  the  hot  and  dry  summer  and  fall  months,  the  intermittent 
areas  of  brush  occurring  throughout  the  site  become  a  critical  fire  hazard. 

The  underlying  bedrock  within  the  former  Camp  SLO  site  area  is  intensely  folded,  fractured,  and 
faulted.  The  site  is  underlain  by  a  mixture  of  metamorphic,  igneous,  and  sedimentary  rocks  less 
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than  200  million  years  old.  Scattered  throughout  the  site  are  areas  of  fluvial  sediments  overlaying 
metamorphosed  material  known  as  Franciscan  melange.  These  areas  are  intruded  by  plugs  of 
volcanic  material  that  comprise  a  chain  of  former  volcanoes  extending  from  the  southwest 
portion  of  the  site  to  the  coast.  Due  to  its  proximity  to  the  tectonic  interaction  of  the  North 
American  and  Pacific  crustal  plates,  the  area  is  seismically  active.  Additional  details  are 
available  in  Reference  1 1 . 


4,2.3  MUNITIONS  CONTAMINATION 


A  large  variety  of  munitions  have  been  reported  as  used  at  the  former  Camp  SLO.  Munitions 
debris  from  the  following  sources  was  observed  throughout  MRS  05  during  the  2007  SI: 


•  4.2-inch  white  phosphorus  mortar 

•  4.2-inch  mortar  base  plate 

•  3. 5 -inch  rocket 

•  37mm  projectile 

•  75  mm  projectile 

•  flares  found  of  newer  metal;  suspected 
from  CNG  activities 


•  105mm  projectile 

•  60mm  mortar 

•  81mm  mortar 

•  Practice  bomb 

•  30  cal.  casings  and  fuzes 


At  the  particular  site  of  this  demonstration,  60-mm,  81 -mm,  and  4.2-in  mortars  and  mortar 
fragments  have  been  observed.  During  the  initial  EM61  Mkll  cart  survey,  two  2.36-in  rockets 
were  found  on  the  surface.  The  excavation  of  two  50’  x  50’  grids  in  October  2008,  as  part  of  the 
preparatory  activities,  has  confirmed  these  observations  and  provided  information  on  the  depths 
of  munitions  at  this  target  site. 


4,2,4  SITE  CONFIGURATION 


The  11.8-acre  demonstration  site  is  shown  in  Figure  4-2  as  a  series  of  30m  x  30m  cells  with  a 
topographical  map  as  the  background.  The  cells  are  color-coded  based  on  the  data  collection 
systems  which  were  deployed  on  them,  tan  color  for  all  systems  and  blue  for  vehicular  systems 
only.  The  site  spans  a  significant  fraction  of  the  hillside  that  is  the  historical  mortar  target.  The 
test  pit  was  located  near  the  logistics  base  and  the  calibration  strip  was  located  outside  the  inner 
fence  line,  convenient  to  the  site  access  road. 
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Figure  4-2  -  ESTCP  UXO  Classification  Study  demonstration  site  at  the  former  Camp  San  Luis  Obispo. 
The  site  is  shown  as  a  series  of  30m  x  30m  cells.  See  the  text  for  further  discussion. 


5.0  TEST  DESIGN 

5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

Each  demonstration  was  designed  to  be  exeeuted  in  two  stages.  The  first  stage  eonsisted  of  a 
standard  MTADS  dynamic  survey  of  the  site.  For  the  APG  demonstration,  the  MTADS 
magnetometer  array  was  the  survey  instrument.  The  details  of  the  magnetometer  survey  can  be 
found  in  Reference  5.  For  the  former  Camp  SFO  demonstration,  the  MTADS  EM61  Mkll  array 
was  the  survey  instrument.  The  details  of  the  Mkll  survey  ean  be  found  in  Reference  12.  The 
choice  of  the  appropriate  technology  for  the  first  stage  of  the  survey  is  governed  by  the 
combination  of  site  history  (expected  TOls  and  clutter)  and  opportunity  costs.  The  APG 
Standardized  Test  Site  had  recently  completed  a  reeonfiguration  in  April  2008  and  NRF  was 
requested  to  eonduet  a  magnetometer  survey  of  the  entire  APG  Standardized  UXO  Test  Site. 
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Past  experience  at  APG  and  with  the  seeded  TOIs  indicated  that  the  magnetometer  survey  would 
provide  acceptable  detection  for  the  later  TEMTADS  survey.  For  the  former  Camp  SLO 
demonstration,  both  the  MEADS  magnetometer  and  EM61  Mkll  array  were  deployed  to  the  site 
and  available.  Based  on  characterization  measurements  of  both  the  site  geology  and  the  TOIs 
with  both  arrays,  the  EM61  Mkll  array  data  had  a  significantly  higher  SNR  for  detection  of  the 
known  TOIs,  so  it  was  selected. 

Anomaly  locations  were  identified  from  the  survey  data  in  a  combination  automated  /  manual 
method.  A  data  segment  around  each  anomaly  center  was  extracted  and  analyzed  using  the  UX- 
Analyze  subsystem  of  the  Oasis  montaj  software  package  to  fit  the  data  to  a  dipole  model  and 
extract  the  associated  fit  parameters  (position,  depth,  equivalent  size).  These  fit  results 
constituted  the  source  anomaly  list  for  the  second  stage  of  each  demonstration. 

This  method  relies  on  the  establishment  of  an  anomaly  detection  threshold.  At  the  former  Camp 
Sibert  demonstration  site,  a  single  munitions  type  was  present  [13].  Pit  measurements  at  various 
depths  and  orientations  of  an  example  article  were  made  and  bounding  response  curves 
generated  for  the  4.2-in  mortar,  the  munitions  of  interest.  The  anomaly  detection  threshold  was 
then  set  based  on  the  least-favorably  predicted  response  at  the  USACoE  standard  llx  depth. 
These  demonstration  sites  each  contained  different  mixes  of  emplaced  munitions  and  suspected 
existing  munitions  contamination.  Individual  anomaly  detection  thresholds  were  established  for 
each  site/area  based  on  sets  of  pit  measurements  made  for  each  of  the  emplaced  items.  For  each 
site/area,  the  smallest  appropriate  least-favorable  response  was  used  to  determine  the  threshold. 
The  details  of  the  anomaly  selection  process,  including  the  response  curves  can  be  found  in  the 
demonstration  data  reports  [5,  12]. 

The  second  stage  of  each  demonstration  was  the  survey  of  each  site/area  using  the  TEMTADS 
array  developed  as  part  of  ESTCP  Project  MR-0601.  The  array  was  positioned  roughly  over  the 
center  of  each  anomaly  on  the  source  anomaly  list  and  a  data  set  collected.  Each  data  set  was 
then  inverted  using  the  data  analysis  methodology  discussed  in  Section  6.3,  estimated  target 
parameters  determined,  and  ultimately  a  classification  made  for  each  anomaly.  The  resulting 
prioritized  dig  lists  were  then  submitted  to  either  the  ATC  or  IDA  for  scoring  and  performance 
assessment. 

The  schedule  of  field  testing  activities  is  provided  in  Figure  5-1  as  a  Gantt  chart. 
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Activity  Name 
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' 
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■ 
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■ 

SLO  TEMTADS  Data 
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May 
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Jul 
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Sept 

Oct 

Nov 
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Mar 

Apr 

May 

Jun 

Figure  5-1  -  Schedule  of  field  testing  activities 

5.2  SITE  PREPARATION 

Each  demonstration  site  had  been  previously  configured  with  clearly-marked  calibration  and 
open  field  scenarios.  At  least  one  GPS  control  point  was  provided  at  each  site.  Basic  facilities 
such  as  portable  toilets  and  field  buildings  were  provided  at  APG  and  acquired  for  SLO.  Secure 
storage  for  larger  vehicles  and  sensor  arrays  was  limited  at  both  sites.  A  40-foot  shipping 
container  was  mobilized  to  each  site  for  the  duration  of  each  demonstration  to  provide 
convenient,  secure  storage  for  the  MTADS  tow  vehicle  and  the  sensor  trailer.  The  container  was 
removed  at  the  end  of  each  demonstration. 

5.3  SYSTEMS  SPECIFICATION 

This  demonstration  was  conducted  using  the  NRL  MTADS  tow  vehicle  and  subsystems.  The 
tow  vehicle  and  each  subsystem  are  described  further  in  the  following  sections. 

5.3.1  MTADS  Tow  Vehicle 

The  MTADS  has  been  developed  by  the  NRL  Chemistry  Division  with  support  from  ESTCP. 
The  MTADS  hardware  consists  of  a  low-magnetic-signature  vehicle  that  is  used  to  tow  the 
different  sensor  arrays  over  large  areas  (10-25  acres  /  day)  to  detect  buried  UXO.  The  MTADS 
tow  vehicle  and  TEMTADS  array  are  shown  in  Eigure  5-2. 
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Figure  5-2  -  MTADS  tow  vehicle  and  TEMTADS  array. 

5.3.2  RTK  GPS  System 

Positioning  is  provided  using  cm-level  Real  Time  Kinematic  (RTK)  Global  Positioning  System 
(GPS)  receivers.  To  achieve  cm-level  precision,  a  fixed  reference  base  station  is  placed  on  an 
established  first-order  survey  control  point  near  the  survey  area.  The  base  station  transmits 
corrections  to  the  GPS  rover  at  1  Hz  via  a  radio  link. 

The  TEMTADS  array  is  located  in  three-dimensional  space  using  a  three-receiver  RTK  GPS 
system  shown  schematically  in  Figure  2-3  [14].  The  three-receiver  configuration  extends  the 
concept  of  RTK  operations  from  that  of  a  fixed  base  station  and  a  moving  rover  to  moving  base 
stations  and  moving  rovers.  The  lead  GPS  antenna  (and  receiver,  Main)  receives  corrections 
from  the  fixed  base  station  at  1  Hz.  The  corrected  position  of  the  Main  GPS  antenna  is  reported 
at  10-20  Hz.  The  Main  receiver  also  operates  as  a  ‘moving  base,’  transmitting  corrections  (by 
serial  cable)  to  the  next  GPS  receiver  (AVRl)  which  uses  the  corrections  to  operate  in  RTK 
mode. 

A  vector  (AVRl,  heading  (yaw),  angle  (pitch),  and  range)  between  the  two  antennae  is  reported 
at  10  Hz.  AVRl  also  provides  ‘moving  base’  corrections  to  the  third  GPS  antenna  (AVR2)  and  a 
second  vector  (AVR2)  is  reported  at  10  Hz.  All  GPS  measurements  are  recorded  at  full  RTK 
precision,  ~2-5  cm.  The  GPS  position  is  averaged  for  2  seconds  as  part  of  the  data  acquisition 
cycle.  The  averaged  position  and  orientation  information  are  then  recorded  to  the  position  data 
file. 

5.3.3  Time-Domain  Electromagnetic  Sensor 

The  TEMTADS  array  is  a  5  x  5  square  array  of  individual  sensors.  Each  sensor  has  dimensions 
of  40  cm  X  40  cm,  for  an  array  of  2  m  x  2  m  overall  dimensions.  The  rationale  of  this  array 
design  is  discussed  in  Reference  15.  The  result  is  a  cross-track  and  down-track  separation  of  40 
cm.  Sensor  numbering  is  indicated  in  Figure  2-3.  The  transmitter  electronics  and  the  data 
acquisition  computer  are  mounted  in  the  tow  vehicle.  Custom  software  written  by  NRE  provides 
both  navigation  to  the  individual  anomalies  and  data  acquisition  functionality.  After  the  array  is 
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positioned  roughly  centered  over  each  anomaly,  the  data  acquisition  cycle  is  initiated.  Each 
transmitter  is  fired  in  a  sequence  winding  outward  from  the  center  position  (12)  in  a  clockwise 
direction.  The  received  signal  is  recorded  for  all  25  Rx  coils  for  each  transmit  cycle.  The 
transmit  pulse  waveform  duration  is  2.7s.  While  it  is  possible  to  record  the  entire  decay  transient 
at  500  MHz,  we  have  found  that  binning  the  data  into  115  time  gates  simplifies  the  analysis  and 
provides  additional  signal  averaging  without  significant  loss  of  temporal  resolution  in  the 
transient  decays  [16].  The  data  are  recorded  in  a  binary  format  as  a  single  file  with  25  data 
points  (one  data  point  per  Tx  cycle). 

5.4  DATA  COLLECTION 

5.4.1  Scale  of  Demonstration 

The  APG  demonstration  was  conducted  at  the  APG  Standardized  UXO  Test  Site.  A 
magnetometer  survey  was  conducted  on  the  Calibration  and  Blind  Grids,  as  well  as  the  Indirect 
Fire  Area  (approximately  4.3  acres)  on  May  7,  2008.  The  TEMTADS  array  surveyed  the 
Calibration  and  Blind  Grids.  The  array  was  also  deployed  to  approximately  700  anomalies  in  the 
Indirect  Fire  Area  that  were  detected  from  the  magnetometer  data  set.  The  TEMTADS  array 
portion  of  the  demonstration  occurred  from  June  16-23,  2008. 

A  cued  discrimination  survey  of  the  former  Camp  SEO  demonstration  site  was  conducted  within 
the  11.8  acre  final  demonstration  site  of  approximately  1,500  previously-identified  anomalies 
from  the  anomaly  list  generated  from  the  MTADS  EM61  Mkll  data  set.  This  survey  was 
conducted  using  the  NRL  TEMTADS.  The  MTADS  EM61  Mkll  data  collection  occurred  during 
the  period  May  11  -  18,  2009.  The  TEMTADS  demonstration  occurred  during  the  period  June  8 
-  18,2009. 

5.4.2  Sample  Density 

Magnetometer  data  were  collected  with  nominal  down-track  spacing  of  6  cm  and  cross  track 
spacing  of  25  cm.  EM61  Mkll  EMI  data  were  collected  with  a  nominal  down-track  spacing  of 
15  cm  and  a  cross  track  spacing  of  50  cm.  Two  orthogonal  surveys  were  conducted  to  increase 
target  illumination  and  data  density.  The  EMI  data  spacing  for  the  TEMTADS  is  fixed  at  40  cm 
in  both  directions  by  the  array  design. 

5.4.3  Quality  Checks 

Since  the  TEMTADS  operates  in  a  cued  mode,  the  data  QC  procedures  and  checks  differ  from 
that  of  survey  mode  instruments.  The  status  of  the  RTK  GPS  system  can  be  visually  verified  by 
the  operator  prior  to  starting  the  data  collection  cycle,  assuring  that  the  position  and  orientation 
information  are  valid  (FQ  3,  PDOP  <  4)  during  the  collection  period. 

Two  data  quality  checks  were  performed  on  the  TEMTADS  data.  After  background  subtraction, 
contour  plots  of  the  signal  were  generated  for  the  25  transmit/receive  pairs  at  a  decay  time  of 
0.042  ms.  An  example  of  a  good  data  set  from  a  single  anomaly  with  a  large  SNR  is  shown  in 
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Figure  5-3  for  the  APG  Calibration  Area  item  16.  The  plots  were  visually  inspeeted  to  verily  that 
there  was  a  well  defined  anomaly  without  extraneous  signals  or  dropouts.  QC  on  the 
transmit/receive  cross  terms  was  based  on  the  dipole  inversion  results.  Our  experience  has 
shown  that  data  glitches  show  up  as  reduced  dipole  fit  coherence. 


Figure  5-3  -  Monostatic  QC  contour  plot  for  Calibration  Area  item  16. 

The  issue  of  when  multiple  objects  are  found  to  be  under  the  array  simultaneously,  generating 
overlapping  signatures,  can  also  be  addressed  at  this  point  in  the  data  QC  process.  An  example 
case  is  shown  in  Figure  5-4.  There  are  two  apparent  issues  in  the  data  set.  First,  there  appears  to 
be  a  small,  shallow  bit  of  scrap  on  top  of  the  target.  Second,  there  was  a  bit  of  scrap  present  in 
the  background  file  used.  This  latter  issue  is  seen  in  the  data  from  element  0.  The  fit  coherence 
using  all  elements  is  0.652.  If  elements  0,5,10,11,15,16,17,20,21,22  are  excluded,  the  fit 
coherence  for  the  remaining  elements  is  0.985  and  the  betas  match  our  library  105mm  HEAT 
betas  very  well. 
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1.0 


Figure  5-4  -  Monostatic  QC  contour  plot  for  example  anomaly. 

Any  data  set  that  was  deemed  unsatisfactory  by  the  data  analyst,  was  flagged  and  not  processed 
further.  The  anomaly  corresponding  to  the  flagged  data  was  logged  for  future  re-acquisition. 
Data  which  meet  these  standards  are  of  the  quality  typical  of  the  MTADS  system. 

5,4,4  Data  Summary 

The  primary  performance  metrics  for  this  demonstration  are  the  classification  performance 
results  for  the  TEMTADS  array  at  the  two  demonstration  sites.  The  first  demonstration  was 
conducted  at  the  APG  Standardized  UXO  Test  Site.  The  performance  results  are  provided  by  the 
site  managers  after  the  classification  rankings  are  submitted  [8].  The  ground  truth  of  this  site  is 
held  by  the  Pis  and  results  are  only  reported  in  aggregate.  For  the  former  Camp  SLO 
demonstration,  there  is  no  single  set  of  performance  results  as  the  UXO  Classification  Study 
involves  multiple  analysis  of  each  collected  data  set.  See  the  UXO  Classification  Study  for 
former  Camp  SLO  Final  Report  [17]  for  results  by  combination. 

5,5  VALIDATION 


5,5,1  Aberdeen  Proving  Grounds 

With  the  exception  of  the  Calibration  Grid,  the  ground  truth  for  the  Standardized  sites  is  held 
back  from  individual  technology  demonstrators  to  preserve  the  utility  of  the  Blind  Grid  and  Open 
Field  Areas.  Results  from  the  Blind  Grid  and  the  Indirect  Fire  Area  were  submitted  to  ATC  for 
performance  evaluation.  Scoring  results  have  been  received  and  are  available  [8]. 
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5,5,2  Former  Camp  San  Luis  Obispo 


At  the  conclusion  of  data  collection  activities,  all  anomalies  on  the  master  anomaly  list 
assembled  by  the  Program  Office  were  excavated.  Each  item  encountered  was  identified, 
photographed,  its  depth  measured,  its  location  determined  using  cm-level  GPS,  and  the  item 
removed  if  possible.  All  non-hazardous  items  were  saved  for  later  in-air  measurements  as 
appropriate.  This  ground  truth  information,  once  released,  was  used  to  validate  the  objectives 
listed  in  Section  3.0. 

6.0  DATA  ANALYSIS  AND  PRODUCTS 

The  TEMTADS  is  a  cued  UXO  classification  system.  There  were  two  parts  to  the  data 
collection  in  each  demonstration:  an  initial  survey  which  served  primarily  to  locate  targets,  and 
TEMTADS  measurements  made  over  the  each  target  detected  in  the  survey.  The  TEMTADS 
data  were  used  for  target  classification. 

6,1  PREPROCESSING 

The  collected  survey  data  were  preprocessed  on  site  for  quality  assurance  purposes  using 
standard  TEMTADS  procedures  and  checks  as  discussed  in  Section  5.4.3  and  References  5  and 

6. 

The  TEMTADS  has  25  transmitters/receiver  pairs.  Eor  each  transmit  pulse,  the  response 
measured  at  all  of  the  receivers  was  recorded  simultaneously.  Eor  each  target,  a  25  x  25  x  N  data 
array  is  generated,  where  N  is  the  number  of  recorded  time  gates.  The  current  system 
configuration  bins  the  data  in  121  logarithmically  spaced  gates.  During  the  preprocessing  step, 
the  recorded  signals  are  normalized  by  the  transmitter  currents  to  account  for  any  transmitter 
variations.  To  account  for  time  delay  due  to  effects  of  the  receive  coil  and  electronics,  we 
subtract  0.028  ms  from  the  nominal  gate  times  [18].  The  delay  was  previously  determined 
empirically  by  comparing  measured  responses  for  test  spheres  with  theory.  The  measured 
responses  include  distortions  due  to  transmitter  ringing  and  related  artifacts  out  to  about  40  ps. 
Consequently  we  only  included  the  responses  from  decay  times  beyond  40  ps  in  our  analyses. 
This  leaves  115  gates  spaced  logarithmically  between  0.042  ms  and  25  ms. 

The  background  response  was  subtracted  from  each  target  measurement  using  data  collected  in  a 
nearby  target-free  region,  or  background.  Background  locations  were  selected  from  quiet  areas 
observed  in  the  survey  data.  All  of  the  background  measurements  were  inter-compared  to 
evaluate  background  variability  and  to  identify  outlier  measurements  potentially  corresponded  to 
measurements  over  non-ferrous  targets.  We  have  not  observed  significant  background 
variability  in  our  measurements  at  our  Blossom  Point  test  site,  and  were  able  to  use  blank  ground 
measurements  from  100  meters  away  for  background  subtraction  on  targets  in  the  test  field. 

Geo-referencing  of  the  array  data  is  based  on  the  GPS  data,  which  gives  the  location  of  the  center 
of  the  array  and  the  orientation  of  the  array.  Sensor  locations  within  the  array  are  fixed  by  the 
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array  geometry.  Dipole  inversion  of  the  array  data  (Section  6.3)  determines  target  location  in 
local  array-based  coordinates.  The  local  position  was  then  transformed  to  absolute  coordinates 
using  the  array  location  and  orientation  determined  from  the  corresponding  GPS  data. 

6.2  TARGET  SELECTION  FOR  DETECTION 

Targets  were  selected  from  the  survey  data  using  the  threshold  exceedance  methods  described  in 
References  5  and  12  using  a  physics-based  threshold  in  Oasis  montaj.  The  data  chips  associated 
with  the  detected  anomalies  were  then  processed  in  an  automatic  processing  mode  of  UX- 
Analyze.  The  results  of  the  automatic  processing  were  then  reviewed  in  UX-Analyze’s 
interactive  mode  to  allow  operator  experience  to  be  included  in  the  target  selection.  In  the  case 
of  the  APG  demonstration,  a  small  number  of  additional  anomalies  were  identified  by  the 
operator  and  added  to  the  anomaly  list.  For  the  former  Camp  SLO  demonstration,  the  operator 
coalesced  double-peaked  features  into  a  single  anomaly  where  appropriate. 

6.3  PARAMETER  ESTIMATION 

The  raw  signature  data  from  the  TEMTADS  reflect  details  of  the  sensor/target  geometry  as  well 
as  inherent  EMI  response  characteristics  of  the  targets  themselves.  In  order  to  separate  out  the 
intrinsic  target  response  properties  from  sensor/target  geometry  effects  we  invert  the  signature 
data  to  estimate  principal  axis  magnetic  polarizabilities  for  the  targets.  The  TEM  data  are 
inverted  using  the  standard  induced  dipole  response  model  wherein  the  effect  of  eddy  currents 
set  up  in  the  target  by  the  primary  field  is  represented  by  a  set  of  three  orthogonal  magnetic 
dipoles  at  the  target  location  [19]. 

Given  a  set  of  measurements  of  the  target  response  with  varying  geometries  or  "look  angles"  at 
the  target,  the  data  can  be  inverted  to  determine  the  (X,Y,Z)  location  of  the  target,  the  orientation 
of  its  principal  axes  ((t),0,\|/),  and  the  principal  axis  polarizabilities  (Pi,P2,P3).  The  basic  idea  is  to 
search  out  the  set  of  nine  parameters  (X,Y,Z,(t),0,\|/,Pi,P2,P3)  that  minimizes  the  difference 
between  the  measured  responses  and  those  calculated  using  the  dipole  response  model. 

Eor  the  TEMTADS  data,  inversion  is  accomplished  by  a  two-stage  method.  In  the  first  stage,  the 
target’s  (X,Y,Z)  dipole  location  is  solved  for  non-linearly.  At  each  iteration  within  this 
inversion,  the  nine  element  polarizability  tensor  (B)  is  solved  linearly.  The  non-linear  inversion 
is  done  simultaneously  over  all  time  gates,  such  that  the  dipole  (X,Y,Z)  location  applies  to  all 
decay  times.  At  each  time  gate,  the  eigenvalues  and  angles  are  extracted  from  the  polarizability 
tensor.  In  the  second  stage,  six  parameters  are  used:  the  three  spatial  parameters  (X,Y,Z)  and 
three  angles  representing  the  yaw,  pitch,  and  roll  of  the  target  (Euler  angles  (t),0,\i/).  In  this 
second  stage  both  the  target  location  and  its  orientation  are  required  to  remain  constant  over  all 
time  gates.  The  value  of  the  best  fit  X,  Y,  and  Z  from  the  first  stage,  and  the  median  value  of  the 
first-stage  angles  are  used  as  an  initial  guess  for  this  stage.  Additional  loops  over  depth  and 
angles  are  included  to  better  ensure  finding  the  global  minimum. 

Not  every  target  from  the  detection  surveys  had  a  strong  enough  TEM  response  to  support 
extraction  of  target  polarizabilities.  All  of  the  data  were  run  through  the  inversion  routines,  and 
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the  results  were  manually  sereened  to  identify  those  targets  that  eould  not  be  reliably  elassified. 
Several  eriteria  were  used  in  this  proeess:  signal  strength  relative  to  baekground,  dipole  fit  error 
(differenee  between  data  and  model  fit  to  data),  the  visual  appearanee  of  the  polarizability 
curves. 

For  moderate-to-good  SNR,  our  derived  polarizabilities  can  be  directly  compared  with  those  of 
our  target  library.  In  these  cases,  a  metric  was  computed  based  on  how  well  the  amplitude  and 
the  ratio  of  the  target  polarizabilities  match  those  of  library  objects.  For  anomalies  in  the  APG 
demonstration  with  lower  SNR  or  for  cases  where  a  visual  inspection  suggested  a  match,  but  the 
match  quality  was  below  our  cutoff,  target  classification  were  done  using  library  matching 
procedures  similar  to  those  used  by  Sky  Research,  Lawrence  Berkeley  Lab  and  ourselves  in  the 
ESTCP  Discrimination  Study  Pilot  Program  at  the  former  Camp  Sibert  [20].  The  fit  quality  of 
an  unconstrained  dipole  inversion  of  the  TEMTADS  data  was  compared  to  the  fit  quality  of  a 
dipole  fit  constrained  by  a  set  of  principal  axis  polarizabilities  drawn  from  a  signature  library. 

The  scoring  rules  for  the  SEO  demonstration  were  different  than  those  used  for  APG 
submissions,  including  the  introduction  of  the  “Can’t  Decide”  category.  Therefore  a  similar,  but 
modified  process  was  used.  The  primary  library  match  algorithm  compares  the  polarizabilities 
of  an  unknown  target  with  each  library  entry  based  on  3  criteria:  the  amplitude  of  Pi,  and  the  two 
shape  parameters,  P2/P1  and  P3/P1.  Eor  each  anomaly,  this  match  was  performed  on  both  the 
polarizabilities  from  the  single  dipole  fit,  and  those  of  each  target  obtained  by  a  multi-dipole  fit. 
The  metric  was  set  to  whichever  set  of  P’s  produces  the  best  match. 

6.4  CLASSIFIER  AND  TRAINING 

Prior  to  the  demonstrations  we  collected  training  data  in  air  for  all  of  the  14  standard  APG 
ordnance  targets  and  the  four  known  munitions  types  for  the  former  Camp  SEO  site.  These  data 
were  used  to  generate  the  fit  library  entries.  Many  of  the  targets  are  composites  of  two  or  more 
distinct  parts,  like  a  steel  body  combined  with  an  aluminum  tail  assembly.  Depending  on  the 
distance  between  the  sensors  and  the  target,  such  items  can  exhibit  a  range  of  slightly  different 
EMI  signatures  corresponding  to  excitation  from  different  directions.  We  included 
measurements  with  the  target  oriented  nose  up,  towards  the  sensor  array,  nose  down,  away  from 
the  array,  flat  and  obliquely  in  the  fit  library.  We  have  assembled  a  fairly  extensive 
polarizability  database  for  clutter  items  recovered  from  several  different  sites.  This  library  was 
used  as  training  data  for  establishing  UXO/clutter  discrimination  boundaries  on  the  direct  match 
metric  and  on  the  coherence  ratio. 

6.5  DATAPRODUCTS 

The  primary  data  products  generated  are  the  ranked  dig  lists  submitted  for  scoring.  Additionally, 
the  archival  data  sets,  stored  individually  by  site  and  by  area  are  provided  for  future  data  analysis 
/  classification  efforts. 


27 


7.0  PERFORMANCE  ASSESSMENT 


The  TEMTADS  array  was  constructed  in  2007,  field  tested  at  the  APG  Standardized  UXO  Test 
Site  in  June  2008  [5,  8],  and  participated  in  the  ESTCP  UXO  Classification  Study  at  the  former 
Camp  SEO  site  in  June  2009  [6].  Due  to  the  nature  of  the  demonstrations  at  each  site  and  the 
employed  scoring  methods,  separate  Performance  Objectives  were  used  for  each  demonstration. 
In  this  Section,  as  in  Section  3.0,  the  performance  results  are  discussed  as  a  whole. 

The  performance  objectives  for  these  demonstrations  were  summarized  in  Table  3-1.  The 
analyses  which  were  used  to  evaluate  these  criteria  are  discussed  in  the  following  sections.  The 
ATC  scoring  report  for  the  demonstration.  Reference  8,  provides  the  source  material  for 
evaluating  several  of  the  performance  objectives. 

7.1  OBJECTIVE:  CALIBRATION  STRIP  RESULTS 

This  objective  supports  that  each  sensor  system  is  in  good  working  order  and  collecting 
physically  valid  data  each  day.  A  calibration  strip,  comprised  of  two  samples  of  each  of  the  four 
targets  of  interest  plus  two  shotputs  (Table  7-1),  was  emplaced  on  site  as  a  means  of  verifying 
proper  system  operation  on  a  daily  basis.  The  strip  was  surveyed  twice  daily,  once  at  the 
beginning  of  the  day,  and  once  at  the  end. 

Table  7-1  -  Details  of  former  Camp  SLO  Calibration  Strip 


Item 

ID 

Description 

Easting  (m) 

Northing  (m) 

Depth  (m) 

Inclination 

Azimuth 
(°cw  from  N) 

T-OOl 

shotput 

705,417.00 

3,913,682.00 

0.25 

N/A 

N/A 

T-002 

81mm 

705,420.92 

3,913,687.63 

0.30 

Vertical  Down 

0 

T-003 

81mm 

705,424.10 

3,913,692.95 

0.30 

Horizontal 

120 

T-004 

60mm 

705,427.53 

3,913,698.54 

0.30 

Vertical  Down 

0 

T-005 

60mm 

705,430.85 

3,913,704.10 

0.30 

Horizontal 

120 

T-006 

4.2”  mortar 

705,434.54 

3,913,709.44 

0.30 

Vertical  Down 

0 

T-007 

4.2”  mortar 

705,437.99 

3,913,715.04 

0.30 

Horizontal 

120 

T-008 

2.36”  rocket 

705,441.46 

3,913,720.24 

0.30 

Vertical  Down 

0 

T-009 

2.36”  rocket 

705,445.00 

3,913,725.91 

0.30 

Horizontal 

120 

T-OlO 

shotput 

705,448.50 

3,913,731.50 

0.35 

N/A 

N/A 

Prior  to  our  demonstration  at  the  former  Camp  SLO,  we  obtained  measurements  of  each  of  the 
four  munitions  types  with  the  TEMTADS  array.  Using  the  derived  polarizabilities  from  these 
measurements  and  the  forward  model  of  our  TEMTADS  code,  response  curves  were  generated 
for  each  munition.  These  curves  plot  the  minimum  expected  peak  signal  (the  peak  signal  when 
the  target  is  oriented  horizontally)  and  the  maximum  expected  peak  (the  peak  signal  when  the 
target  is  oriented  horizontally)  from  each  target  as  a  function  of  distance  below  sensor  or  depth 
below  ground.  The  peak  signal  is  that  obtained  from  the  monostatic  term  of  the  center  element 
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of  the  array  at  the  first  time  gate.  The  measured  peak  signals  for  the  60mm  mortars  are  given  in 
Figure  7-1  as  an  example. 


Figure  7-1  -  Peak  signals  compared  with  response  curve  for  a  60-mm  mortar. 

This  objective  was  not  successfully  met  in  full.  The  maximum  (red)  and  minimum  (blue) 
response  curves  are  plotted.  For  self-consistency,  we  have  plotted  the  measurements  at  the  mean 
inverted  depth,  rather  than  the  reported  depths.  The  measured  values  generally  fit  well  within 
the  bounding  curves,  with  the  4.2-in  mortar  and  shotput  results  the  poorest,  with  a  tendency  to 
underestimate  the  peak  value. 

Careful  examination  of  the  data  shows  that  this  variation  is  the  result  of  the  shot-to-shot 
precision  with  which  the  array  can  be  positioned  in  exactly  the  same  spot  each  time.  Because  the 
response  curves  are  generated  assuming  the  target  is  directly  below  the  sensor,  any  offset  in  the 
sensor  position  will  result  in  the  derived  peak  signal  being  smaller  than  that  predicted  by  the 
curve,  as  is  observed.  For  future  demonstrations,  the  metric  of  polarizability  amplitude  is 
recommended  as  these  values  are  invariant  to  array  position. 

Table  7-2  shows  the  mean  and  standard  deviation  in  the  peak  measured  signal  for  all  of  the 
emplaced  Calibration  Strip  items.  Of  the  10  items,  three  give  RMS  variations  above  our  stated 
goal  of  15%,  with  only  T-008  being  significantly  above.  This  target  consistently  inverted  to  a 
depth  of  18.5  cm,  and  thus  showed  the  largest  spatial  variation  in  signal.  This,  coupled  with  the 
array  not  always  being  properly  centered  on  the  target,  explains  the  larger  variation  for  this 
object. 
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Table  7-2  -  Peak  Signals  for  former  Camp  SLO  Calibration  Strip  Emplaced  Items 


Item  ID 

Description 

Depth  (m) 

Mean  Signal  (mV/Amp) 

Std  Dev. 
(mV/Amp,In) 

Variation 

(%) 

T-OOl 

shotput 

0.25 

27.44 

4.18 

15.23 

T-002 

81mm 

0.30 

15.46 

1.39 

8.99 

T-003 

81mm 

0.30 

10.96 

0.91 

8.30 

T-004 

60mm 

0.30 

8.74 

0.89 

10.18 

T-005 

60mm 

0.30 

6.73 

1.13 

16.79 

T-006 

4.2”  mortar 

0.30 

52.38 

5.82 

11.11 

T-007 

4.2”  mortar 

0.30 

41.75 

5.25 

12.57 

T-008 

2.36”  rocket 

0.30 

32.70 

7.91 

24.19 

T-009 

2.36”  rocket 

0.30 

4.04 

0.32 

7.92 

T-OlO 

shotput 

0.35 

11.82 

1.52 

12.86 

The  variability  and  accuracy  of  the  positional  fit  parameters  for  the  Calibration  Strip  emplaced 
items  were  determined  by  comparing  the  inverted  Northing  and  Easting  values  with  reported 
values.  These  numbers  are  shown  in  Table  7-3.  We  give  the  mean  vector  offset  dR  between  the 
inverted  and  reported  position  as  well  as  the  Easting  (dx)  and  Northing  (dy)  components.  The  dx 
and  dy  values  are  computed  using  the  inverted  positions  minus  the  reported  ones. 

Two  points  are  clear  from  the  values  in  Table  7-3.  Eirst,  the  inversion  process  is  very  robust, 
with  no  standard  deviations  larger  than  2cm.  Second,  there  are  a  few  large  discrepancies 
between  our  inverted  positions  and  the  reported  ones.  All  offsets  are  below  our  target  value  of 
25cm,  with  the  exception  of  T-002.  It  is  possible  that  some  items  have  drifted  or  settled  since 
their  original  emplacement. 

Table  7-3  -  Position  Accuracy  and  Variability  for  former  Camp  SLO  Calibration  Strip  Emplaced  Items 


Item 

ID 

Description 

Depth 

(m) 

Mean  dR 
(m) 

Std  Dev 
dR 

(m,  lo) 

Mean 

dx 

(m) 

Std  Dev 

dx 

(m) 

Mean 

dy 

(m) 

Std  Dev 

dy 

(m) 

T-OOl 

shotput 

0.25 

0.209 

0.007 

0.167 

0.013 

0.124 

0.011 

T-002 

81mm 

0.30 

0.413 

0.008 

-0.372 

0.008 

-0.178 

0.010 

T-003 

81mm 

0.30 

0.058 

0.014 

0.028 

0.010 

-0.049 

0.020 

T-004 

60mm 

0.30 

0.026 

0.013 

0.013 

0.010 

0.015 

0.019 

T-005 

60mm 

0.30 

0.038 

0.012 

-0.003 

0.011 

0.035 

0.017 

T-006 

4.2”  mortar 

0.30 

0.098 

0.008 

0.082 

0.009 

0.054 

0.009 

T-007 

4.2”  mortar 

0.30 

0.063 

0.008 

0.061 

0.009 

0.013 

0.010 

T-008 

2.36”  rocket 

0.30 

0.035 

0.010 

-0.027 

0.011 

0.018 

0.011 

T-009 

2.36”  rocket 

0.30 

0.112 

0.016 

0.051 

0.010 

-0.099 

0.018 

T-OlO 

shotput 

0.35 

0.166 

0.006 

0.133 

0.008 

-0.098 

0.014 

7.2  OBJECTIVE:  REDUCTION  OF  FALSE  ALARMS 

This  is  the  primary  measure  of  the  effectiveness  of  this  technology.  By  collecting  high-quality, 
precisely-located  data,  we  expect  to  be  able  to  discriminate  munitions  from  scrap  and  frag  with 
high  efficiency.  At  a  seeded  test  site  such  as  the  APG  standardized  test  site,  the  metric  for  false 
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alarm  elimination  is  straightforward.  We  prepared  a  ranked  dig  list  for  the  targets  we 
interrogated  with  a  dig/no-dig  threshold  indieated  and  ATC  personnel  use  their  automated 
seoring  algorithms  to  assess  our  results. 

This  Objeetive  was  sueeessfully  met.  The  TEMTADS  surveyed  anomalies  deteeted  by  the 
MTADS  magnetometer  system  in  the  Blind  Grid  and  Indireet  Fire  Areas.  For  the  Blind  Grid 
Test  Area,  the  diserimination  stage  results  are  summarized  in  Table  7-4  and  Table  7-5  (subsets  of 
Table  6a  of  Referenee  8),  broken  out  by  munitions  type  and  emplaeement  depth.  For  the  Indireet 
Fire  Test  Area,  the  diserimination  stage  results  are  summarized  in  Table  7-6  and  Table  7-7 
(subsets  of  Table  6e  of  Referenee  8),  broken  out  by  munitions  type  and  emplaeement  depth.  The 
Diserimination  Stage  Probability  of  Deteetion  is  defined  as  the  number  of  eorreetly 

identified  munitions  divided  by  the  number  of  emplaeed  munitions,  and  the  eorresponding 
Probability  of  False  Positive  is  the  number  of  elutter  items  ineorreetly  identified  as 

munitions  divided  by  the  number  of  emplaeed  elutter  items. 

Table  7-4  -  TEMTADS  Blind  Grid  Test  Area  Results 


T>  disc 

Ad 

All  Types 

105-mm 

81/60mm 

37/25-mm 

Munitions 

Scores 

0.97 

0.93 

0.97 

1.00 

0  to  4D " 

1.00 

1.00 

1.00 

1.00 

4D  to  8D 

1.00 

1.00 

1.00 

1.00 

8Dto  12D 

0.67 

0.67 

0.00 

1.00 

Burial  depths  are  reported  in  units  of  the  munitions’  outer  diameter,  or  D 


Table  7-5  -  TEMTADS  Blind  Grid  Test  Area  Ffp"*'""  Results 


Tt  disc 
rfp 

All  Masses 

0  to  0.25  kg 

>0.25  to  1 
kg 

>1  to  10  kg 

All  Depths 

0.01 

0.02 

0.00 

0.00 

0  to  0.15m 

0.01 

0.02 

0.00 

0.00 

0.15  to 

0.3m 

0.00 

0.00 

0.00 

0.00 

0.3  to  0.6m 

N/A 

N/A 

N/A 

N/A 

Table  7-6  -  TEMTADS  Indirect  Fire  Test  Area  Results 


T>  disc 

Ad 

All  Types 

105-mm 

81/60mm 

37/25-mm 

Munitions 

Scores 

0.92 

0.93 

0.93 

0.91 

By  Density 

High 

0.88 

0.92 

0.91 

0.80 

Medium 

0.94 

0.97 

0.89 

0.97 

Low 

0.94 

0.90 

0.97 

0.94 

By  Depth 

0  to  4D 

0.96 

0.94 

0.97 

0.97 

4D  to  8D 

0.92 

0.94 

0.92 

0.86 

8D  to  12D 

0.72 

0.75 

0.78 

0.67 
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Table  7-7  -  TEMTADS  Indirect  Fire  Test  Area  Results 


Tt  disc 
ffp 

All  Masses 

0  to  0.25  kg 

>0.25  to  1 
kg 

>1  to  10  kg 

All  Depths 

0.04 

0.03 

0.02 

0.11 

0  to  0.15m 

0.04 

0.04 

0.02 

0.13 

0.15  to 

0.3m 

0.04 

0.00 

0.06 

0.06 

0.3  to  0.6m 

0.08 

0.00 

0.00 

0.20 

Discrimination  Efficiency  (E)  and  Ealse  Positive  Rejeetion  Rate  (Rfp)  measure  the  effeetiveness 
of  the  diserimination  stage  proeessing.  The  goal  of  diserimination  is  to  retain  the  greatest 
number  of  munitions  detections  from  the  anomaly  list,  while  rejeeting  the  maximum  number  of 
anomalies  arising  from  non-munitions  items.  Effieieney  measures  the  fraetion  of  deteeted 
munitions  retained  after  diserimination,  while  the  rejeetion  rate  measures  the  fraetion  of  false 
alarms  rejected.  The  measures  are  defined  relative  to  the  number  of  munitions  items  or  the 
number  of  elutter  items  that  were  aetually  deteeted  by  the  sensor.  The  results  for  the  Blind  Grid 
and  Indireet  Eire  Test  Areas  are  summarized  in  Table  7-8  and  Table  7-9,  from  Tables  7a  and  7e 
of  Reference  8.  Performanee  levels  are  shown  at  two  speeifie  operating  points  on  the  ROC 
eurve;  one  at  the  point  where  no  deerease  in  Pd  is  ineurred  and  the  other  at  the  operator-seleeted 
operating  point  or  threshold.  In  the  Blind  Grid,  98%  of  the  emplaeed  munitions  items  were 
deteeted.  Of  these,  99%  (97%  of  the  emplaced  munitions)  were  eorrectly  elassified  at  our 
seleeted  operating  point,  with  a  eorresponding  false  positive  rejeetion  rate  of  99%.  Moving  out 
the  ROC  to  the  point  where  100%  of  the  detected  munitions  items  are  eorreetly  elassified 
reduees  the  false  positive  rejection  rate  to  95%.  In  the  Indireet  Eire  Area,  94%  of  the  emplaeed 
munitions  were  deteeted,  and  92%  of  the  emplaced  munitions  were  eorreetly  elassified,  resulting 
in  a  diserimination  effieieney  of  98%.  The  eorresponding  false  positive  rejeetion  rate  was  92%. 

Table  7-8  -  TEMTADS  Blind  Grid  Test  Area  Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False  Positive 
Rejection 
Rate 

At  Operating  Point 

0.99 

0.99 

With  No  Loss  of 

Pd 

1.00 

0.95 

Table  7-9  -  TEMTADS  Indirect  Fire  Test  Area  Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False  Positive 
Rejection 
Rate 

At  Operating  Point 

0.98 

0.92 

With  No  Loss  of 

Pd 

1.00 

0.58 
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7.3  OBJECTIVE:  LOCATION  AND  DEPTH  ACCURACY 


An  important  measure  of  how  effieiently  any  required  remediation  will  proeeed  is  the  aeeuraey 
of  predieted  loeation  of  the  targets  marked  to  be  dug.  Large  loeation  errors  lead  to  eonfusion 
among  the  UXO  techs  assigned  to  the  remediation  costing  time  and  often  leading  to  removal  of  a 
small,  shallow  object  when  a  larger,  deeper  object  was  the  intended  target.  The  average  error 
and  deviation  in  both  horizontal  axes  was  computed  for  the  items  which  are  selected  for 
excavation  during  the  validation  phase  of  the  demonstration.  We  provided  an  estimated  position 
for  all  targets  we  interrogated  to  the  appropriate  personnel  at  each  site  they  used  their  scoring 
algorithms  to  assess  our  results. 

This  Objective  was  not  successfully  met  in  full.  For  the  APG  demonstration,  the  location 
accuracy  of  fit  parameters  generated  from  the  TEMTADS  array  data  are  given  in  Table  7-10  and 
Table  7-11,  taken  from  Tables  9a  and  9c  of  Reference  8.  Horizontal  errors  are  not  calculated  for 
the  Blind  Grid. 

Table  7-10  -  TEMTADS  Blind  Grid  Test  Area  Location  Error  and  Standard  Deviation 


Mean 

(m) 

Standard 

Deviation 

(m) 

Northing 

N/A 

N/A 

Easting 

N/A 

N/A 

Depth 

0.02 

0.04 

Table  7-1 1  -  TEMTADS  Indirect  Fire  Test  Area  Location  Error  and  Standard  Deviation 


Mean 

(m) 

standard 

Deviation 

(m) 

Northing 

0.01 

0.05 

Easting 

0.01 

0.05 

Depth 

0.00 

0.06 

For  the  former  Camp  SLO  demonstration,  the  average  Northing  position  error  for  all  measured 
data  was  1.5  cm,  while  the  average  Easting  position  error  was  3  cm.  The  standard  deviations, 
however,  were  larger  than  desired,  with  both  values  about  25  cm.  Excluding  those  anomalies  for 
which  multiple  targets  were  found  produces  negligible  improvement.  We  suspect  the  higher 
values  are  due  to  the  large  number  of  small,  low  SNR  clutter  items,  which  result  in  greater 
uncertainty  in  both  the  measured  and  fitted  values.  Indeed,  if  we  restrict  ourselves  to  those 
anomalies  which  we  classified  as  ‘likely  UXO,’  the  Northing  and  Easting  standard  deviations 
drop  to  7cm  and  5cm,  respectively.  The  average  depth  error  for  all  measured  data  was  2.5cm. 
The  standard  deviation  was  a  bit  larger  than  desired,  with  a  value  of  14cm.  Excluding  those 
anomalies  for  which  multiple  targets  were  found  and  restricting  the  analysis  to  those  anomalies 
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classified  as  ‘likely  UXO’  to  exclude  low  SNR  clutter  items,  the  standard  deviation  reduees  to 
7om. 


8.0  COST  ASSESSMENT 

The  costs  for  a  3,000  anomaly  cued  survey  using  the  TEMTADS  are  detailed  in  Table  8-1.  The 
eosts  assoeiated  with  the  former  Camp  SLO  demonstration  were  used  as  the  basis  for  this 
assessment.  The  eosts  reported  here  do  not  inelude  any  eosts  assoeiated  with  the  reeonnaissanee 
data  that  was  used  to  identify  and  ereate  the  eued  list  of  anomalies.  Implieit  in  our  assessment  is 
that  that  the  3,000  anomalies  are  bounded  within  an  area  of  roughly  25  aeres,  or  120 
anomalies/aere.  Survey  time  would  inerease  inerementally  as  the  anomaly  density  deereased  as 
transit  time  between  anomalies  inereased.  All  eosts  are  estimated  in  FY  2009  dollars. 

8.1  COST  MODEL 

A  eost  model  was  eonstructed  from  the  eosts  assoeiated  with  the  TEMTADS  survey  of  the 
former  Camp  SEO  demonstration  site.  Cost  eategories  were  mobilization,  logisties  (if  required), 
field  work,  demobilization,  data  analysis,  and  reporting.  Based  on  the  demonstrated  produetion 
rate  for  the  former  Camp  SEO  demonstration,  four  weeks  of  field  work  would  be  required. 
Based  on  the  produetion  rate  for  the  APG  demonstration,  the  SLO  produetion  rate  is  a 
eonservative  lower  bound.  This  model  does  not  inelude  the  eost  of  anomaly  olassifieation  and 
produeing  a  prioritized  dig  list. 
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Table  8-1  -  Summary  of  Costs  for  a  25-Acre,  3000  Anomaly  TEMTADS  Survey. 


Cost  Category 


Mobilization  Costs 


Logistics 


(if  required) 


Operating  Costs 


(4-week  Survey) 


Analysis  &  Reporting 


Demobilization  Costs 


Total  Cost 


Sub-Category  Cost 


Preliminary  Site  Visit 

$6,500 

Test  Plan  Preparation 

$10,250 

Equipment  Prep  and  Packing 

$9,750 

53’  Trailer  Transportation 

$5,800 

Analysts  Set-up 

$10,000 

Outbound  Travel  for  3  Personnel 

$2,250 

Establish  GPS  Control  Points 

$2,000 

Delivery/Removal  of  Logistics  Items 

$2,000 

Rental  of  Logistics  Items  /  Fuel 

$6,000 

Materials 

$6,500 

Supervisor 

$19,000 

On-site  Analyst 

$31,800 

Vehicle  Operator 

$12,750 

Field  Technician  (2  each) 

$17,250 

Per  Diem  (3  Personnel  x  4  weeks  x  $1250) 

$15,000 

Rental  Vehicles 

$8,650 

System  Maintenance 

$13,000 

Sensor  Repair 

$21,600 

Data  Reduction  to  Anomaly  List 

$28,000 

Demonstration  Data  Report 

$16,750 

Equipment  Unpacking 

$3,500 

53’  Trailer  Transportation 

$5,800 

Inbound  Travel  for  3  Personnel 

$2,250 

Sub-Total 

$44,550 


$16,500 


$139,050 


$44,750 

$11,550 


$256,400 
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8.2  COST  DRIVERS 


Two  factors  were  expeeted  to  be  strong  drivers  of  eost  for  this  teehnology  as  demonstrated.  The 
first  is  the  number  of  anomalies  whieh  ean  be  surveyed  per  day.  Higher  produetivity  in  data 
colleetion  equates  to  more  anomalies  investigated  for  a  given  period  of  time  in  the  field.  The 
time  required  for  analyzing  individual  anomalies  ean  be  signifieantly  higher  than  for  other,  more 
traditional  methods  and  eould  beeome  a  eost  driver  due  to  the  time  involvement.  The  thoughtful 
use  of  available  automation  teehniques  for  individual  anomaly  analysis  with  operator  QC  support 
can  moderate  this  effect. 

8.3  COST  BENEFIT 

The  main  benefit  to  using  a  UXO  olassifieation  proeess  is  eost-related.  The  ability  to  reduce  the 
number  of  non-hazardous  items  that  have  to  be  dug  or  dug  as  hazardous  direetly  reduees  the  eost 
of  a  remediation  effort.  The  additional  information  for  anomaly  classifieation  provided  by  the 
TEMTADS  array  provides  additional  information  for  the  purposes  of  anomaly  olassifieation.  If 
there  is  buy-in  from  the  stakeholders  to  use  these  teehniques,  this  information  ean  be  used  to 
reduoe  oosts. 

As  an  example  of  the  potential  cost  benefit  of  using  this  technology  on  an  actual  cleanup,  an 
example  soenario  is  presented.  The  two  demonstrations  disoussed  in  this  report  were  both  of 
short  duration  with  a  small  number  of  anomalies  to  oapitalize  mobilization  oosts  aoross. 
Therefore,  considering  only  the  field  work  and  data  analysis  costs  given  in  Table  8-1,  a 
reasonable  estimate  of  the  cost/anomaly  for  deploying  the  TEMTADS  array  long  term  is 
$61/anomaly.  The  staffing  level  assumed  in  Table  8-1  is  appropriate  for  a  new  R&D  projeot 
demonstration  and  eould  be  reduoed  to  support  sustained  produotion  operations. 

Similarly,  the  cost  of  fielding  an  appropriately  certified  UXO  dig  team,  without  mobilization 
oosts,  ean  range  between  37  and  50  k$  per  week  (EY2010  dollars).  Assuming  that  the  team  ean 
olear  between  310  and  420  anomalies  a  day,  the  eost  to  dig  an  anomaly  is  90  -  160  $/anomaly. 
Assuming  that  one  peroent  of  the  items  dug  are  in  faot  UXO,  the  remediation  of  those  UXO  must 
be  aocounted  for  and  a  remediation  eost  of  SlkAJXO,  the  average  eost  per  dig  would  range  from 
100  -  170  $/anomaly. 

Einally,  for  the  example,  assume  a  hypothetieal  aetual  eleanup  site  with  10,000  anomalies  to  be 
eleared.  Using  the  above  analysis,  the  eost  of  the  eleanup  with  all  anomalies  dug  would  range 
from  1,000  to  1,700  k$  total.  If  one  assumes  that  the  TEMTADS  elassifies  the  anomalies 
suffieiently  well  to  reduoe  the  number  of  actual  digs  required  to  10%  of  the  original  number,  the 
oombined  eost  of  the  TEMTADS  survey  and  the  resultant  digging  drops  to  713  -  783  k$  total, 
for  a  potential  savings  of  29  -  54%  overall. 

9.0  IMPLEMENTATION  ISSUES 

Implementation  issues  for  this  system  /  teehnology  fall  into  two  oategories:  operational  concerns 
and  data  quality  /  analysis  issues.  In  terms  of  operational  eoneerns,  the  TEMTADS  as 
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implemented  is  a  large,  vehiele-towed  system  that  operates  best  in  large,  open  areas.  As  seen  at 
the  former  Camp  SLO  demonstration,  it  is  possible  to  operate  the  system  in  roeky  terrain  with 
grades  approaehing  20%,  but  at  redueed  operating  eapaeity  and  inereased  system  wear.  Smaller 
versions  of  the  system  are  eurrently  under  development  under  several  ESTCP  and  SERDP 
projeets  to  address  these  eoneems.  The  goal  is  to  design  and  field  units  more  amenable  to 
operation  in  more  eonfined  terrain  and  topology  and  smaller  tow  vehieles  /  man-portable  and 
handheld  operation.  Another  serious  limitation  will  be  anomaly  density.  Eor  all  sensors,  there  is 
a  limiting  anomaly  density  above  whieh  the  response  of  individual  targets  eannot  be  separated. 
We  have  ehosen  relatively  small  sensors  for  this  array  whieh  should  help  with  this  problem  but 
we  eannot  eliminate  it.  Based  on  experiments  at  our  test  pit  at  Blossom  Point,  the  results  of  this 
demonstration,  and  work  done  on  the  former  Camp  Sibert  data  sets,  anomaly  densities  of  300 
anomalies/aere  or  higher  would  limit  the  applieability  of  this  system  as  more  than  20%  of  the 
anomalies  would  have  another  anomaly  within  a  meter. 

In  terms  of  data  quality,  one  pays  a  small  penalty  in  signal  amplitude  to  use  the  smaller 
TEMTADS  sensor  eoil  as  eompared  to  other  systems  sueh  as  a  high-power  EM61  Mkll.  The 
dramatieally  improved  eleetrieal  performanee  of  these  new  sensors  helps  eounterbalanee  this 
issue,  partieularly  in  the  ability  to  perform  better  averaging  (or  staeking).  However,  one  needs  in 
plaee  robust  data  QC  teehniques  to  know  when  to  employ  these  eapabilities  in  an  effieient 
manner. 
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703-696-3826  (V) 
703-696-2114  (E) 
anne.andrews@osd.mil 

Deputy 
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ESTCP 

Dr.  Herb 

Nelson 

ESTCP  Program  Office 

901  North  Stuart  Street, 
Suite  303 

Arlington,  VA  22203 

703-696-3726  (V) 
703-696-2114  (E) 
202-215-4844  (C) 
herbert.nelson@osd.mil 

Program 
Manager,  MM 

Mr.  Peter 
Knowles 

HydroGeoEogic,  Inc. 

11107  Sunset  Hills  Road, 
Suite  400 

Reston,  VA  20190 

703-736-4511  (V) 
pkno  wles@hgl  .com 

Program 
Manager’s 
Assistant,  MM 

Dr.  Dan 
Steinhurst 

Naval  Research  Eab 
Chemistry  Division 

Code  6110 

Washington,  DC  20375 

202-767-3556  (V) 
202-404-8119  (E) 
703-850-5217  (C) 
dan.steinhurst@nrl.navy.mil 

Co-PI  and 

Data  Analyst 

Dr.  Tom  Bell 

SAIC 

200  12th  Street  South 
Arlington,  VA  22202 

703-414-3904  (V) 
703-413-0505  (E) 
301-712-7021  (C) 
thomas.h.bell@saic.com 

Co-PI  and 
Quality 

Control 

Officer 

Mr.  Rick  Fling 

Aberdeen  Test  Center 

410-278-2999  (V) 
301-992-9080  (C) 
rick.  fling@us .  army .  mil 

Test  Site 
Manager 

Mr.  David 
Ragsdale 

California  Polytechnic 

State  University 

San  Euis  Obispo,  CA 

93407 

805-756-6662  (V) 
805-756-1602  (E) 
dragsdal@calpoly.edu 

Environmental 
Health  & 

Safety 

Manager  / 

Risk 

Management 
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